[Hindsight] No output

Michael Trinkala mtrinkala at mozilla.com
Tue Nov 8 02:49:29 UTC 2016


It seems like the output is not being flushed to disk. When it is running
for the first time take a look at output/input/0.log.  I suspect the file
size will show 0 until HS is shutdown and the file is closed (or you send
more data through).  If you see the file update and HS is still not picking
it up that will also be informative. I am not seeing the problem on any of
my Linux boxes so I suspect it is Docker related.

Trink

On Mon, Nov 7, 2016 at 4:10 PM, Thota, Madhukar <
madhukar.thota at libertymutual.com> wrote:

> Hi trink,
>
>
>
> I am running hindsight as Docker image. When I ran the container first
> time, I don’t see any output, but if I stop and start the container, I see
> all messages from previous run.
>
>
>
> *hindsight.cp* file:
>
>
>
> _G['analysis->output.heka_debug'] = '0:0'
>
> _G['analysis'] = '0:0'
>
> _G['input->analysis0'] = '0:0'
>
> _G['input->output.heka_debug'] = '0:0'
>
> _G['input'] = '0:12600'
>
>
>
> *hindsight.tsv*
>
>
>
> Plugin  Inject Message Count    Inject Message Bytes    Process Message
> Count
>
> input.tcp       200     11600   0       0       161647  224615  0       0
>
> output.heka_debug       0       0       0       0       42876   49050   0
>
>
>
> *hindsight stdout:*
>
>
>
> 1478563528815493029 [info] hindsight starting
>
> 1478563528825646642 [info] input.tcp starting
>
> 1478563528828448499 [info] analysis_plugins starting thread: 0
>
> 1478563528837132261 [info] output.heka_debug starting
>
>
>
>
>
> *From: *Michael Trinkala <mtrinkala at mozilla.com>
> *Date: *Monday, November 7, 2016 at 9:20 AM
> *To: *Madhukar Thota <madhukar.thota at LibertyMutual.com>
> *Cc: *"hindsight at mozilla.org" <hindsight at mozilla.org>
> *Subject: *Re: [Hindsight] No output
>
>
>
> I would need to see the Hindsight log output and the hindsight.cp file.
> Best guess: If you have been testing and manually deleting stuff out of the
> output directory make sure you also delete the hindsight.cp file.
>
> Trink
>
>
>
> On Sun, Nov 6, 2016 at 8:30 PM, Thota, Madhukar <
> madhukar.thota at libertymutual.com> wrote:
>
> Hi
>
>
>
> I am playing with hindsight to test simple flow using tcp input and
> heka_debug.lua in output. I can see my messages are coming to tcp input and
> but it’s not going to output.
>
>
>
> I am using Master branch for hindsight, lua_sandbox and extensions.
>
>
>
> Here is my flow:
>
>
>
> *hindsight.cfg:*
>
>
>
> output_path             = "output"
>
> output_size             = 1024 * 1024 * 1024
>
> sandbox_load_path       = nil -- disable dynamic loading
>
> sandbox_run_path        = "/etc/hindsight/run"
>
> sandbox_install_path    = "/usr/share/luasandbox/sandboxes/heka"
>
> analysis_lua_path       = "/usr/lib/luasandbox/modules/?.lua"
>
> analysis_lua_cpath      = "/usr/lib/luasandbox/modules/?.so"
>
> io_lua_path             = analysis_lua_path ..  ";/usr/lib/luasandbox/io_
> modules/?.lua"
>
> io_lua_cpath            = analysis_lua_cpath .. ";/usr/lib/luasandbox/io_
> modules/?.so"
>
>
>
> input_defaults = {
>
> }
>
> analysis_defaults = {
>
> }
>
> output_defaults = {
>
> }
>
>
>
> *Input:  tcp.cfg*
>
>
>
> filename = "tcp.lua"
>
> instruction_limit = 0
>
> address = "0.0.0.0"
>
> port = 5555
>
> default_headers = nil
>
> decoder_module = "decoders.payload"
>
>
>
> *Ouptut: heka_debug.cfg*
>
>
>
> filename        = "heka_debug.lua"
>
> message_matcher = "TRUE"
>
>
>
> In hindsight.tsv file I can four message but zero output.
>
>
>
> Plugin     Inject Message Count        Inject Message Bytes        Process
> Message Count    Process Message Failures Current Memory                Max
> Memory       Max Output         Max Instructions Message Matcher Avg
> (ns)              Message Matcher SD (ns)         Process Message Avg (ns)
> Process Message SD (ns)   Timer Event Avg (ns)          Timer Event SD (ns)
>
> input.tcp               *4*              228         0
> 0              116095  162396  0              0              0
> 0              0              0              0                0
>
> output.heka_debug           0              0              0
> 0              42836    49050    0              72           0
> 0              0              0                8566       1028
>
>
>
>
>
> Any help is appreciated.
>
>
>
>
> _______________________________________________
> Hindsight mailing list
> Hindsight at mozilla.org
> https://mail.mozilla.org/listinfo/hindsight
>
>
>
> _______________________________________________
> Hindsight mailing list
> Hindsight at mozilla.org
> https://mail.mozilla.org/listinfo/hindsight
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.mozilla.org/pipermail/hindsight/attachments/20161107/c5f9d941/attachment.html>


More information about the Hindsight mailing list