Issue 3261 - Introduce module adblockplus::telemetry

Note that it may be worth integrating with some static web::server repository at some point, or redirecting to an adblockplus.org/$path, instead of displaying the welcome-to-nginx page by default. That would also allow to display some static default content when a data submission has succeeded.

[mathias, 2015-11-03 10:34:53]

https://codereview.adblockplus.org/29329669/diff/29329670/hiera/roles/telemet...
File hiera/roles/telemetryserver.yaml (right):

https://codereview.adblockplus.org/29329669/diff/29329670/hiera/roles/telemet...
hiera/roles/telemetryserver.yaml:7: location: /submit/filter-hit-statistics

Note that this location conflicts with the one initially suggested (/submit)
which, however, does not make much sense in the context of an explicit
telemetry.adblockplus.org domain which may host multiple endpoints - alongside
some other stuff - in the future.

To name a particular example: It could make sense to move the Adblock Browser
feedback endpoint to the telemetry.adblockplus.org domain as well, i.e.
/submit/adblock-browser-feedback. The current endpoint at adblockbrowser.org
could become a forwarding proxy until it does not receive requests any more.

[Fred, 2015-11-05 13:59:48]

LGTM

https://codereview.adblockplus.org/29329669/diff/29329670/modules/adblockplus...
File modules/adblockplus/manifests/telemetry.pp (right):

https://codereview.adblockplus.org/29329669/diff/29329670/modules/adblockplus...
modules/adblockplus/manifests/telemetry.pp:12: #   A hash of
adblockplus::telemetry::endpoin setup parameters to become
Typo: endpoint

https://codereview.adblockplus.org/29329669/diff/29329670/modules/adblockplus...
modules/adblockplus/manifests/telemetry.pp:27: #       # see
adblockplus::telementry::endpoint
Typo: telemetry

[Felix Dahlke, 2015-11-10 15:29:14]

https://codereview.adblockplus.org/29329669/diff/29329670/hiera/roles/telemet...
File hiera/roles/telemetryserver.yaml (right):

https://codereview.adblockplus.org/29329669/diff/29329670/hiera/roles/telemet...
hiera/roles/telemetryserver.yaml:7: location: /submit/filter-hit-statistics
On 2015/11/03 10:34:53, mathias wrote:
> 
> Note that this location conflicts with the one initially suggested (/submit)
> which, however, does not make much sense in the context of an explicit
> http://telemetry.adblockplus.org domain which may host multiple endpoints -
alongside
> some other stuff - in the future.
> 
> To name a particular example: It could make sense to move the Adblock Browser
> feedback endpoint to the http://telemetry.adblockplus.org domain as well, i.e.
> /submit/adblock-browser-feedback. The current endpoint at
http://adblockbrowser.org
> could become a forwarding proxy until it does not receive requests any more.

Well, feedback and telemetry are quite different things, I'd prefer to keep this
separated. If we were to combine it I think we would need a more generic
umbrella, e.g. "submit.adblockplus.org". (IIRC, Mozilla has "api.mozilla.org"
for a similar purpose.)

Also: How I remember the discussion, we would send _all_ telemetry data within
that one JSON, i.e. "/submit" seems fine. But let's double check that that's
really how we discussed it should work.

[mathias, 2015-11-10 15:33:00]

On 2015/11/10 15:29:14, Felix Dahlke wrote:
> Well, feedback and telemetry are quite different things, I'd prefer to keep
this
> separated. If we were to combine it I think we would need a more generic
> umbrella, e.g. "submit.adblockplus.org". (IIRC, Mozilla has "api.mozilla.org"
> for a similar purpose.)
> 
> Also: How I remember the discussion, we would send _all_ telemetry data within
> that one JSON, i.e. "/submit" seems fine. But let's double check that that's
> really how we discussed it should work.

IIRC we did not explicitly address this topic. I just figured during
implementation that it may make sense to adjust these bits. Though the example
may not be the best one, I still believe it's reasonable to adjust the location
accordingly - maybe using another path than the one suggested though.

[Felix Dahlke, 2015-11-12 07:07:37]

On 2015/11/10 15:33:00, mathias wrote:
> On 2015/11/10 15:29:14, Felix Dahlke wrote:
> > Well, feedback and telemetry are quite different things, I'd prefer to keep
> this
> > separated. If we were to combine it I think we would need a more generic
> > umbrella, e.g. "submit.adblockplus.org". (IIRC, Mozilla has
"api.mozilla.org"
> > for a similar purpose.)
> > 
> > Also: How I remember the discussion, we would send _all_ telemetry data
within
> > that one JSON, i.e. "/submit" seems fine. But let's double check that that's
> > really how we discussed it should work.
> 
> IIRC we did not explicitly address this topic. I just figured during
> implementation that it may make sense to adjust these bits. Though the example
> may not be the best one, I still believe it's reasonable to adjust the
location
> accordingly - maybe using another path than the one suggested though.

I really don't think we should be making separate requests for different kinds
of telemetry data sent from the same client. We could get Wladimir in the loop
to see if he disagrees, but I personally doubt that.

The format suggested in https://issues.adblockplus.org/ticket/394 is definitely
generic enough, all the filter stats are in the "filters" field, and we can add
additional arbitrary fields for whatever else we want to send.

[mathias, 2015-11-12 10:53:41]

On 2015/11/12 07:07:37, Felix Dahlke wrote:
> I really don't think we should be making separate requests for different kinds
> of telemetry data sent from the same client.

That is true, but that is *not* the actual intention I try to address here --

We will make separate requests for whatever kind of telemetry data, if we ever
decide to send telemetry data not only in the context of the browser extensions.

[Felix Dahlke, 2015-11-13 07:38:17]

On 2015/11/12 10:53:41, mathias wrote:
> On 2015/11/12 07:07:37, Felix Dahlke wrote:
> > I really don't think we should be making separate requests for different
kinds
> > of telemetry data sent from the same client.
> 
> That is true, but that is *not* the actual intention I try to address here --
> 
> We will make separate requests for whatever kind of telemetry data, if we ever
> decide to send telemetry data not only in the context of the browser
extensions.

Well, it's true that the data for our browsers might end up looking differently.
/submit/adblockplus then? That way we could use a different URL where we send
different data, e.g. /submit/adblockbrowser, or /submit/adblockbrowserandroid
even (that's a good example since I believe we want to use the Telemetry stuff
already built into Firefox and expand on that, on Android).

/submit/filter-hit-statistics however doesn't make sense for the reasons above,
it's too specific.

[mathias, 2015-11-13 09:23:38]

On 2015/11/13 07:38:17, Felix Dahlke wrote:
> Well, it's true that the data for our browsers might end up looking
differently.
> /submit/adblockplus then? That way we could use a different URL where we send
> different data, e.g. /submit/adblockbrowser, or /submit/adblockbrowserandroid
> even (that's a good example since I believe we want to use the Telemetry stuff
> already built into Firefox and expand on that, on Android).

Acknowledged.

[Felix Dahlke, 2015-11-13 12:30:57]

LGTM