Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: v2 analytics and split testing #247

Open
wants to merge 19 commits into
base: main
Choose a base branch
from
Open

Conversation

kyle-ssg
Copy link
Member

@kyle-ssg kyle-ssg commented Sep 10, 2024

  • Uses the new v2 analytics endpoint
  • Exposes a trackEvent function, this will let applications record conversion events e.g. checkout and have them recorded in split testing analytics

To opt in, this adds splitTestingAnalytics to flagsmith.init

@kyle-ssg kyle-ssg marked this pull request as draft September 10, 2024 16:36
@kyle-ssg kyle-ssg changed the title feat: v2 analytics feat: v2 analytics and split testing Sep 10, 2024
@kyle-ssg kyle-ssg marked this pull request as ready for review September 10, 2024 19:31
Copy link
Member

@rolodato rolodato left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left some armchair-reviewer comments, I'm really not familiar with this v2 API yet :)

@@ -523,6 +502,7 @@ const Flagsmith = class {
this.withTraits = {}
}
this.identity = userId;
this.events.map(this.trackEvent)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would cause 1 API request per tracked event - is this intentional or batching not supported by the API?

Previously, identify would make at most 1 API call, whereas now it could make any number of calls

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It isn't batched no. These events are specific to conversions e.g. checkout so I don't imagine there'd be a lot

flagsmith-core.ts Outdated Show resolved Hide resolved
* Only available for self hosted split testing analytics.
* Track a conversion event within your application, used for split testing analytics.
*/
trackEvent: (event: string) => Promise<void>;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's safe to assume that events could be recorded at a much higher rate than flags are evaluated, so they should definitely be batched and flushed in the background. The current implementation is probably good enough to validate the idea in a PoC but I would exclude this method from this SDK's public/versioned API until we can implement event batching.

This method being async is also not ergonomic to use on event handlers for DOM events like clicking, hovering or navigating.

Copy link
Member Author

@kyle-ssg kyle-ssg Sep 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't think of a case where we'd track events often, and even if they were it'd happen way less than flag evaluations since they'd occur potentially every render. The only usecase I can imagine is conversions e.g. checkout.

evaluations: Object.keys(evaluations).map((feature_name)=>(
{
feature_name,
"identity_identifier": this.identity||null,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unless I'm missing something, the current this.identity here could be different than the one when the event happened, for example if the user logged out/in before analytics were flushed.

The same applies to hasFeature below - haven't reviewed this in much detail so might be completely off

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm yeah I suppose they should be more of a snapshot, I'll have a think about this / look if there are any nasty race conditions

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants