Software bugs are annoying: Buggy software can bring down anything from your sales to mid-air planes. Debugging software is difficult, time consuming and costs a fortune. A host of frameworks and processes have been created to make software testing easier and ensure that fewer bugs make it into production, and invariably they all fail every now and then.
When this happens, the pain for software developers and the pain for businesses begin. Developers need to find the source code that caused the bug and run it in a test environment that looks as close as possible to production. The situation that caused the bug must also be recreated.
This typically works by adding logging statements and breakpoints in the code, and tracing the execution in the code and its dependencies until the bug is located and can be fixed. Then the new code must be rebuilt and redeployed into production.
Frankly, it’s painful just thinking about it, let alone having to go through it. Or Weis and Liran Haimovitch are two software engineers who have been through this time and time again and felt the pain, so they decided to do something about it.
Get the data, ignore the redeployment
Weis and Haimovitch founded Rookout in 2017, based on a seemingly simple premise: add uninterrupted breakpoints in live code and get real-time data from its real environment, without shutting down applications. It sounds a bit like black magic in software development, so we had to wonder how it worked.
As CEO Weis told ZDNet, the Rookout framework works in two stages. First of all, the framework is integrated into a running application via an SDK. Once complete, instructions are received at the request of the Rookout management system, and bytecode or opcode manipulation is applied on the fly to augment the running service (s) as if it had been deployed. with the additional logging / data collection code first.
So, not exactly magic: there is SDK integration via necessary. But it’s something that should be done once, and Weis said it’s “just a dependency, a 2 minute setup”. What can you do after doing this? Collect data as you wish and send it wherever you want, according to Weis.
Rookout breakpoints can collect any data that could be collected by adding an extra log line in the code: local variables, global variables, stack traces, local thread storage, metrics and whatever else your code is doing. has access. The only exception is return values for method calls, which are not enabled for security reasons.
As for performance, Weis said that adding a log line with Rookout, or doing it by coding and redeploying, creates the same basic end result in memory, so there is no overload. He added that Rookout makes sure to only apply read-only effects and always stay within the resource throttling limits set by the administrator.
Rookout supports JVM programming languages, such as Java, Scala or Groovy, Node.js, and Python, in all clouds, containers, Kubernetes, and serverless. Rookout also works on external and third-party libraries, as long as a filename and line number can be supplied. The collected data points can then be sent wherever they need to go: an application performance monitoring solution, alerting and logging tools, business intelligence tools, and more.
There’s a catch though: at this point it’s not certain that your preferred IDE coding environment is supported by Rookout. You may need to upgrade to Rookout’s own web IDE to use Rookout. Weis said that Rookout is an API-driven infrastructure platform, which means you can connect to it through any interface you want.
Rookout strives to customize several interfaces besides its API as well as part of other products eg Sentry.io + Rookout, Circle-CI. Its current focus, however, is its Web IDE. Weis noted that:
Building our own custom web IDE allows us to create a bespoke experience to suit our unique data collection experience, which would be more difficult to do with the limited power of modern IDEs. That said, allowing developers to never leave their IDEs is a big YES and we plan to provide it very soon for major IDEs like Jetbrains and VScode.
Instant observability for machine learning frameworks
Today, Rookout announced support for two of the most popular open source machine learning frameworks: Apache Spark and Tensorflow. Spark is supported both as a code instrumentation platform and as a data target for the data collected by Rookout. Weis said other related environments such as Jupyter laptops are also supported.
With the major shift towards developing machine learning-based applications, this should come as no surprise. But what can Rookout offer users of the machine learning framework, and how does it work?
Rookout calls the capabilities it adds “instantaneous observability” and notes that by bringing observability to AI, Rookout will allow data scientists to understand, improve, monitor, debug, and iterate faster. on their machine learning (ML) models, develop new data dimensions, add functionality and improve accuracy. Interestingly, Rookout links this to the explainability of AI.
“AI is a black box. You bring in data and you pull out of it, we usually don’t understand why it makes decisions,” said CTO Haimovitch. “Data scientists who develop AI rely on engineering and IT teams to provide them with the data they need to improve their models. It takes time, costs money, and can be extremely frustrating. The data they need is there, but they couldn’t access it. — until now.”
Unexpected behavior could be an error in the structure of the model or a bias in the data, or it could be a classic bug in the wrapping code. Each of them will have their own very different solution. If a model needs more training, it can take weeks of computing time. If the model itself is to be extended, data scientists may have to do complicated design work. On the other hand, a logical error, once found, could be corrected in seconds.
Rookout says it allows data scientists to observe and understand their models while they are running, giving them access to model responses, inputs, and peripheral data on demand, without having to ask the engineering team code changes or wait for the next version.
On the Rookout
Seeing this, we wondered about the fundamentals of the Rookout framework, if it is possible to continue to extend it to more environments, and if they are working with vendors to achieve this. Weis noted that the exact implementation depends on the framework’s runtime environment, but basically Rookout supports most out-of-the-box frameworks without any customization:
Rookout uses some existing thinking / instrumentation capabilities of each runtime environment it attaches to, and also has many proprietary techniques per runtime environment to achieve the right controlled manipulation effect. But of course, some frameworks require special attention (eg AWS Lambda or GitHub Electron).
Some frameworks require us to develop special cases in our code, others require the user to call Rookout in a specific way – for example, by wrapping Lambda functions. So far, we haven’t needed any help from the framework creators to add support. That said, we would be happy to collaborate if it becomes necessary, including contributing to open-source.
Rookout, which filed for patents around its data collection and pipeline technology, came out of stealth in 2018. It secured an initial investment of $ 4.2 million led by TLV Partners and Emerge, has a growing core team of low-level engineers and cybersecurity experts, and is focused on the North American market, with clients such as Backblaze, Two-Sigma, Maverik and Guesty.
And if you’re wondering what’s in a name, Weis said the towers inspired them because they’re smart birds who can use tools and even shape hooks to find things. The founders of Rookout envisioned a bird that sits with developers and serves them whenever they need to get data on the fly from their live code.
If you’re looking to ease the pain of debugging, Rookout may be your thing.