From acb7a57bc154d944a9c54bd06a106850cc073ad8 Mon Sep 17 00:00:00 2001 From: Karen Metts Date: Thu, 19 Mar 2020 14:15:12 -0400 Subject: [PATCH 1/7] Doc clarifications and improvements --- docs/index.asciidoc | 30 +++++++++++++++++++++++++++++- 1 file changed, 29 insertions(+), 1 deletion(-) diff --git a/docs/index.asciidoc b/docs/index.asciidoc index c06e573..873103f 100644 --- a/docs/index.asciidoc +++ b/docs/index.asciidoc @@ -357,7 +357,35 @@ Specify the truststore type here. One of `JKS` or `PKCS12`. Default is `JKS` * There is no default value for this setting. A Hash of urls in this format : `"name" => "url"`. -The name and the url will be passed in the outputed event +The name and the url will be passed in the outputted event. + +The values in urls can be either: + +* a string url (which will be issued as an HTTP GET). +* a sub-hash containing many useful keys provided by the Manticore backend: +** url: the String url +** method: (optional) the HTTP method to use (defaults to GET) +** user: (optional) the HTTP Basic Auth user +** password: (optional) the HTTP Basic Auth password. The password +must be under an auth sub-hash for Manticore, but this plugin accepts it either way. +** headers: a hash containing key-value pairs of headers. The header must be under +an auth sub-hash for Manticore, but this plugin accepts it either way. +** body: a string (supported only on POST and PUT requests) +** possibly other options mentioned in the +(https://www.rubydoc.info/github/cheald/manticore/Manticore/Client#http-instance_method)[Manticore docs]. +Note that these options are not thoroughly tested and therefore liable to +break in unexpected ways if we replace the backend. + +*Notes:* + +* Passwords specified in this way are prone to exposure in plugin log output. +The plugin does not declare them as passwords, and therefore doesn't wrap them in +leak-reducing wrappers as we do elsewhere. +* We don't guarantee that boolean-type options like follow_redirects are supported +correctly. The strings `true` or `false` may get passed through, and in ruby any +string is "truthy." +* Our implementation of this plugin precludes the ability to specify auth[:eager] +as anything other than true [id="plugins-{type}s-{plugin}-validate_after_inactivity"] ===== `validate_after_inactivity` From 53d4fa36287773b95dc273ea1d3a8a0cdc368681 Mon Sep 17 00:00:00 2001 From: Karen Metts <35154725+karenzone@users.noreply.github.com> Date: Fri, 24 Apr 2020 13:44:37 -0400 Subject: [PATCH 2/7] Doc: Correct acceptable url values Co-Authored-By: Ry Biesemeyer --- docs/index.asciidoc | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/index.asciidoc b/docs/index.asciidoc index 873103f..61b5c71 100644 --- a/docs/index.asciidoc +++ b/docs/index.asciidoc @@ -365,11 +365,11 @@ The values in urls can be either: * a sub-hash containing many useful keys provided by the Manticore backend: ** url: the String url ** method: (optional) the HTTP method to use (defaults to GET) -** user: (optional) the HTTP Basic Auth user +** user: (optional) the HTTP Basic Auth user. The user must be under + an auth sub-hash for Manticore, but this plugin also accepts it either way. ** password: (optional) the HTTP Basic Auth password. The password -must be under an auth sub-hash for Manticore, but this plugin accepts it either way. -** headers: a hash containing key-value pairs of headers. The header must be under -an auth sub-hash for Manticore, but this plugin accepts it either way. + must be under an auth sub-hash for Manticore, but this plugin accepts it either way. +** headers: a hash containing key-value pairs of headers. ** body: a string (supported only on POST and PUT requests) ** possibly other options mentioned in the (https://www.rubydoc.info/github/cheald/manticore/Manticore/Client#http-instance_method)[Manticore docs]. From 2c44fbe56d1dab9220c3f8bab49df779c2db296e Mon Sep 17 00:00:00 2001 From: Karen Metts <35154725+karenzone@users.noreply.github.com> Date: Fri, 24 Apr 2020 14:06:38 -0400 Subject: [PATCH 3/7] Doc: Update content to clarify that only the Manticore options aren't thoroughly tested Co-Authored-By: Ry Biesemeyer --- docs/index.asciidoc | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/docs/index.asciidoc b/docs/index.asciidoc index 61b5c71..78be802 100644 --- a/docs/index.asciidoc +++ b/docs/index.asciidoc @@ -372,9 +372,10 @@ The values in urls can be either: ** headers: a hash containing key-value pairs of headers. ** body: a string (supported only on POST and PUT requests) ** possibly other options mentioned in the -(https://www.rubydoc.info/github/cheald/manticore/Manticore/Client#http-instance_method)[Manticore docs]. -Note that these options are not thoroughly tested and therefore liable to -break in unexpected ways if we replace the backend. + (https://www.rubydoc.info/github/cheald/manticore/Manticore/Client#http-instance_method)[Manticore docs]. + Note that Manticore options that are not explicitly documented above are not + thoroughly tested and therefore liable to break in unexpected ways if we + replace the backend. *Notes:* From c228d491bbf0ebaf8a596bcb759fb8fb70834f53 Mon Sep 17 00:00:00 2001 From: Karen Metts <35154725+karenzone@users.noreply.github.com> Date: Fri, 24 Apr 2020 14:12:19 -0400 Subject: [PATCH 4/7] Doc: Correct link formatting inside bulleted list --- docs/index.asciidoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/index.asciidoc b/docs/index.asciidoc index 78be802..5a74504 100644 --- a/docs/index.asciidoc +++ b/docs/index.asciidoc @@ -372,7 +372,7 @@ The values in urls can be either: ** headers: a hash containing key-value pairs of headers. ** body: a string (supported only on POST and PUT requests) ** possibly other options mentioned in the - (https://www.rubydoc.info/github/cheald/manticore/Manticore/Client#http-instance_method)[Manticore docs]. + https://www.rubydoc.info/github/cheald/manticore/Manticore/Client#http-instance_method[Manticore docs]. Note that Manticore options that are not explicitly documented above are not thoroughly tested and therefore liable to break in unexpected ways if we replace the backend. From 16fed9dc9c8bc9b0a73587dcaa7f0da478effdda Mon Sep 17 00:00:00 2001 From: Karen Metts Date: Fri, 24 Apr 2020 14:33:07 -0400 Subject: [PATCH 5/7] Doc:Alphabetize config options and descriptions --- docs/index.asciidoc | 38 +++++++++++++++++++------------------- 1 file changed, 19 insertions(+), 19 deletions(-) diff --git a/docs/index.asciidoc b/docs/index.asciidoc index 5a74504..9e4d2b4 100644 --- a/docs/index.asciidoc +++ b/docs/index.asciidoc @@ -95,8 +95,6 @@ This plugin supports the following configuration options plus the <> |<>|no -| <> |<>|No | <> |<>|No | <> |a valid filesystem path|No | <> |a valid filesystem path|No @@ -109,6 +107,7 @@ This plugin supports the following configuration options plus the <> |<>|No | <> |<>|No | <> |<>|No +| <> |<>|No | <> |<>|No | <> |<>|No | <> |<<,>>|No @@ -121,6 +120,7 @@ This plugin supports the following configuration options plus the <> |<>|No | <> |<>|No | <> |<>|Yes +| <> |<>|No | <> |<>|No |======================================================================= @@ -129,23 +129,6 @@ input plugins.   -[id="plugins-{type}s-{plugin}-user"] -===== `user` - - * Value type is <> - * There is no default value for this setting. - -Username to use with HTTP authentication for ALL requests. Note that you can also set this per-URL. -If you set this you must also set the `password` option. - -[id="plugins-{type}s-{plugin}-password"] -===== `password` - - * Value type is <> - * There is no default value for this setting. - -Password to be used in conjunction with the username for HTTP authentication. - [id="plugins-{type}s-{plugin}-automatic_retries"] ===== `automatic_retries` @@ -249,6 +232,14 @@ If you'd like to work with the request/response metadata. Set this value to the name of the field you'd like to store a nested hash of metadata. +[id="plugins-{type}s-{plugin}-password"] +===== `password` + + * Value type is <> + * There is no default value for this setting. + +Password to be used in conjunction with the username for HTTP authentication. + [id="plugins-{type}s-{plugin}-pool_max"] ===== `pool_max` @@ -388,6 +379,15 @@ string is "truthy." * Our implementation of this plugin precludes the ability to specify auth[:eager] as anything other than true +[id="plugins-{type}s-{plugin}-user"] +===== `user` + + * Value type is <> + * There is no default value for this setting. + +Username to use with HTTP authentication for ALL requests. Note that you can also set this per-URL. +If you set this you must also set the `password` option. + [id="plugins-{type}s-{plugin}-validate_after_inactivity"] ===== `validate_after_inactivity` From 723a446617affe8540a8c230821297e6e89272b1 Mon Sep 17 00:00:00 2001 From: Karen Metts Date: Fri, 24 Apr 2020 14:45:48 -0400 Subject: [PATCH 6/7] Bump to v.5.0.2 --- CHANGELOG.md | 5 ++++- logstash-input-http_poller.gemspec | 2 +- 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 0738160..8fedeb7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,6 @@ +## 5.0.2 + - [DOC]Expanded url option to include Manticore keys [#119](https://github.com/logstash-plugins/logstash-input-http_poller/pull/119) + ## 5.0.1 - Fixed minor doc and doc formatting issues [#107](https://github.com/logstash-plugins/logstash-input-http_poller/pull/107) @@ -18,7 +21,7 @@ - Docs: Remove row in overview table to fix build error ## 4.0.2 - - Don't bleed URLs credentials on startup and on exception #82 + - Don't bleed URLs credentials on startup and on exception [#82](https://github.com/logstash-plugins/logstash-input-http_poller/pull/82) ## 4.0.1 - Fix some documentation issues diff --git a/logstash-input-http_poller.gemspec b/logstash-input-http_poller.gemspec index 06bfc05..b50a5cd 100644 --- a/logstash-input-http_poller.gemspec +++ b/logstash-input-http_poller.gemspec @@ -1,6 +1,6 @@ Gem::Specification.new do |s| s.name = 'logstash-input-http_poller' - s.version = '5.0.1' + s.version = '5.0.2' s.licenses = ['Apache License (2.0)'] s.summary = "Decodes the output of an HTTP API into events" s.description = "This gem is a Logstash plugin required to be installed on top of the Logstash core pipeline using $LS_HOME/bin/logstash-plugin install gemname. This gem is not a stand-alone program" From eb19844e155cc34d75d8859b83ee5fb79cf5bd95 Mon Sep 17 00:00:00 2001 From: Karen Metts <35154725+karenzone@users.noreply.github.com> Date: Tue, 19 May 2020 17:16:05 -0400 Subject: [PATCH 7/7] Apply suggestions from code review Co-authored-by: Ry Biesemeyer --- docs/index.asciidoc | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/index.asciidoc b/docs/index.asciidoc index 9e4d2b4..0fa8372 100644 --- a/docs/index.asciidoc +++ b/docs/index.asciidoc @@ -238,7 +238,7 @@ hash of metadata. * Value type is <> * There is no default value for this setting. -Password to be used in conjunction with the username for HTTP authentication. +Password to be used in conjunction with <> for HTTP authentication. [id="plugins-{type}s-{plugin}-pool_max"] ===== `pool_max` @@ -370,10 +370,10 @@ The values in urls can be either: *Notes:* -* Passwords specified in this way are prone to exposure in plugin log output. +* Passwords specified as a part of `urls` are prone to exposure in plugin log output. The plugin does not declare them as passwords, and therefore doesn't wrap them in leak-reducing wrappers as we do elsewhere. -* We don't guarantee that boolean-type options like follow_redirects are supported +* We don't guarantee that boolean-type options like Manticore's `follow_redirects` are supported correctly. The strings `true` or `false` may get passed through, and in ruby any string is "truthy." * Our implementation of this plugin precludes the ability to specify auth[:eager] @@ -386,7 +386,7 @@ as anything other than true * There is no default value for this setting. Username to use with HTTP authentication for ALL requests. Note that you can also set this per-URL. -If you set this you must also set the `password` option. +If you set this you must also set the <> option. [id="plugins-{type}s-{plugin}-validate_after_inactivity"] ===== `validate_after_inactivity`