D5.1 to D5.6: Tigerbeetle: A tool to compare traces
tigerbeetle is a project aiming at the creation of a fast, flexible trace analysis core. The project name has absolutely no meaning and is subject to change.
This document describes the foundations and ideas behind tigerbeetle:
- easy-to-use
- extensible
- modern
Our goal is to first focus on the LTTng tracer. There are others out there, but LTTng is a powerful, modern, stable and actively maintained kernel and userspace tracer for Linux. Some analysis tools exist for inspecting LTTng traces, but they are either:
- too simple (list of events)
- too old and not maintained anymore
- unresponsive and unstable
Knowing this, here's how we plan to achieve our three core principles.
easy-to-use
From a user's point of view, the trace analysis core should be as easy to use as possible.
As mentioned above, the only thing you should provide to tigerbeetle in order to obtain a decent analysis out of it is a state provider. We won't create and specify a fancy domain-specific language to do this: plain Python 3 FTW.
Using Python gives you, the developer, many advantages:
- easy to use (and learn) scripting language
- access to the Python environment (file opening, database connections, math library, etc.)
- portable state provider
Here are the expected functions a Python state provider should define:
def on_init(state):
# initialize state before receiving any event
def on_event(event, state):
# use event and current state to modify the state
def on_fini(state):
# modify the state one last time after having received all events
That's it.
We choose Python because there's already some work done in this direction using the official LTTng trace reader, Babeltrace. However, other languages could be supported in the future to provide states: Ruby, Lua, Go, Node.js, etc.
If you want faster analysis, a C++11 dynamic library will also work as a state provider, albeit more complex to write and debug.
extensible
Message-oriented middlewares are very trendy recently and we plan to center tigerbeetle around one of them to make it as extensible as possible (between processes and between networked machines).ØMQ will be tried first and kept if it proves to be a good choice.
The goal here is to create an analysis core application which will reply very quickly to asynchronous requests received on a ØMQ socket. Here are a few possible requests:
- I want the event following timestamp T
- I want n events between timestamps A and B
- I want the state value of state attribute a at timestamp T
- I want the state values of state attributes A at timestamp T
- I want to know how many traces are currently being analyzed
- I want to know the begin and end timestamps of the analyzed traces
- etc.
Using ØMQ, any viewer/controller at any place using any language (ØMQ has bindings for almost all modern languages) may be implemented.
Such an application is useless without other applications using its services. The ultimate goal of all this is to facilitate the development of a very responsive Qt 5 trace analyzer, and of a slick web equivalent using a WebSocket-to-ØMQ bridge.
modern
We're not afraid of using cutting edge technologies if this means faster development and more fun to do so without trading off too much performance.
We plan to use:
- GitHub issues/pull requests for contributions
- SCons for the build system
- Travis CI for continuous integration
- C++11 for performance-critical applications/libraries
- Python 3 for pretty much everything else
- ØMQ 4
- Qt 5/PyQt5 for some GUI development
- UTF-8 JSON-RPC for RPCs (requests/replies and notifications)
- YAML for configuration files
All those items also have the advantage of facilitating portability.