Investigating the Potential of AI for Findur JVS Plugin Maintenance

Will ignoring AI leave you stuck in the Jurassic period? Discover how Lucido’s consultancy services can help with Findur JVS plugin maintenance and optimize your system before it goes extinct.

“Your scientists were so preoccupied with whether or not they could, they didn’t stop to think if they should”

Barely a day seems to go by without another news report about AI, and invariably they are not great. Hollywood rightfully struck over the use of generative AI, and you probably deserve what you get if you are a lawyer using it. In our industry, it is a little frustrating that all AI discussion seems to revolve around LLMs (Large Language Models) such as ChatGPT, Copilot, and Gemini, when there are numerous use cases, like the function of Machine Learning, where AI can benefit. That said, we have talked at length about best practices for a Findur support model, so we thought we would investigate if a LLM could help with Findur JVS plugin maintenance. This white paper walks you through what we found.

“It’s a UNIX system! I know this!”

OpenJVS is a Java-based scripting framework allowing for the customization and enhancement of Findur based on pre-defined classes, methods, and APIs. JVS plugins can automate tasks, integrate with other systems, and add new features to the core Findur platform. Given the dynamic nature of markets and API / core code changes with upgrades, these plugins can require regular updates.

Findur JVS plugin maintenance presents several challenges:

Complexity of Code: JVS plugins often involve intricate codebases that require specialized knowledge to update and maintain effectively.
Documentation Gaps: Comprehensive and up-to-date documentation is essential for maintaining these plugins. However, documentation is often outdated or incomplete, making it difficult for new team members to understand and work with the existing code.
Troubleshooting Issues: Identifying and resolving issues within the plugins can be time-consuming and requires deep knowledge of the Findur system and the specific plugin architecture, especially if the logging is poor.
Resource Constraints: Organizations often have limited resources for maintaining and enhancing these plugins or may have no in-house JVS developers.

“God help us, we’re in the hands of engineers”

What if a LLM could help us with this? Ever wondered exactly what generate inventory and position processing do? Or need to fix a plugin that does not compile, but have no access to a JVS developer? We thought we would find out. First, we tested a range of LLMs with the same prompts and ChatGPT-4o’s responses were significantly better than the rest. Our approach then was to simply copy-paste code snippets into ChatGPT and request specific prompts, like asking for documentation, troubleshooting, minor code enhancements, or writing a new plugin from scratch.

To evaluate the results, we asked our team of human JVS developers to assess the output. We looked at the accuracy of ChatGPT’s output, and the relevance of the troubleshooting solutions provided. We compared the reduction in time taken to update documentation, the effectiveness of ChatGPT’s solutions in resolving issues without introducing new errors, and the impact of enhancements on the performance and functionality of the JVS plugins.

Documentation

A straightforward request is to ask ChatGPT to document some of the key processes, like generate inventory and position processing, or where the coding is a little convoluted, like STD_Margin_Call_Process or STD_SWIFT_Cash_Pymt. The results were pretty good! It correctly identified the purpose of the plugins and highlighted the key components. There were a couple of gaps but using it to write the first draft of plugin documentation saves a lot of time.

Next, we added the challenge of updating a short plugin. The code below tests the connection to an external system:

package com.app.process;

import com.app.api.Gateway;
import com.olf.openjvs.*;

public class TestConnection implements IScript
{
        public void execute(IContainerContext context) throws OException
        {       
        try
        {
        Gateway broker = new Gateway();
        if(!broker.testConnection())
                 Util.exitFail("Failed to connect");
                           broker.destroy();
                 }
        catch (Exception e)
        {
                           Util.exitFail("Failed to connect to Vendor: " + e.getMessage());
                 }
        }
}

We asked ChatGPT to provide a summary, and it was able to successfully identify the purpose of the plugin and the external system it connects to (masked here):

This OpenJVS plugin, named TestConnection, is designed to test the connectivity between the system and the Vendor service using the Gateway API.

Summary:

The plugin belongs to the com.wc1.process package and imports necessary classes from com.app.api and com.olf.openjvs.

The class TestConnection implements the IScript interface, making it an executable script within the OpenJVS framework.

The primary method execute(IContainerContext context) contains the logic to test the connection.

The plugin functions as follows:

A Gateway object is instantiated.

The testConnection() method of the Gateway object is called to check the connection.

If the connection test fails, the script terminates with an error message “Failed to connect”.

The broker object is destroyed after the test.

If any exception occurs during the connection test, the script captures the exception and exits with a detailed error message including the exception details.

This plugin ensures that the system can successfully connect to Vendor, handling any failures gracefully by providing clear error messages.

We asked ChatGPT to add a header and comments, as well as make any minor improvements to the code it sees fit. This was the outcome:

package com.app.process;

import com.app.api.Gateway;
import com.olf.openjvs.*;

/**
 * TestConnection Plugin
 *
 * This plugin checks if the system can connect to the Vendor service.
 * It attempts to make a connection and provides a clear message if the connection fails.
 *
 * Author: [Your Name]
 * Date: [Current Date]
 */

public class TestConnection implements IScript {

    /**
     * Executes the script to test the connection to the Vendor service.
     *
     * @param context The container context in which this script is executed.
     * @throws OException If an OpenJVS exception occurs.
     */

    public void execute(IContainerContext context) throws OException {
        Gateway broker = null;
        try {
            // Create a Gateway object to manage the connection
            broker = new Gateway();
       
            // Test the connection to Refinitiv
            if (!broker.testConnection()) {
                // Exit with a failure message if the connection test fails
                Util.exitFail("Failed to connect to Vendor service.");
            }
        } catch (Exception e) {
            // Provide a clear error message if the connection fails
            Util.exitFail("Failed to connect to Vendor: " + e.getMessage());
        } finally {
            // Ensure the broker object is properly closed
            if (broker != null) {
                broker.destroy();
            }
        }
    }
}

It made the following changes:

Header: Added a header to explain the purpose of the script.
Comments: Included comments throughout the script to explain key steps and logic.
Resource Management: Moved the broker.destroy() call into a finally block to ensure that resources are released even if an exception occurs.
Minor Adjustments: Ensured consistent and clear messaging in the error handling and improved code readability.

So far, so good.

Compilation Errors

One client had an old plugin that had been loaded as part of the implementation but never used. They wanted to test it, but it did not compile. When the plugin was pasted into ChatGPT with the compilation errors it identified the following issue:

The errors you’re encountering are due to the PluginLog class not being found. This could be because the PluginLog class is either not available in your environment or not correctly imported. To resolve these errors, you can either ensure the PluginLog class is available and correctly imported, or you can replace the logging mechanism with a standard one provided by OpenJVS.

Here, I’ll replace PluginLog with OConsole for logging, which is part of the OpenJVS package and should be available in your environment.

It then provided the updated code, which compiled successfully in the system. It also identified a deprecated method but was not able to suggest a replacement that worked. While the plugin did compile and run, there was additional work required to achieve the purpose of the script that ChatGPT could not perform.

Enhanced Error Logging

An issue we worked on recently, where we temporarily did not have access to a developer, was a Bloomberg import process that did not generate any meaningful messages when it failed. The process included three plugins (a param script, a main script, and an import utility). ChatGPT was able to parse each plugin in separate prompts and then accurately summarize the overall process and how they fit together.

ChatGPT went through each instance of OException and gave it a unique identifier and error message to more easily identify where the process was failing. It also added that message as an additional column to the Excel file it generated after the process had finished. In a test environment, this all helped with the initial business user analysis, but it would have been significantly quicker to use one of our JVS developers.

New Plugin Development

Another client hit a limitation where the character length in a core field meant that they could not paste the full URL required. As a test, we asked ChatGPT if it was able to write a short plugin to launch the URL from the system. Here is the code it wrote:

package com.custom;

import com.olf.openjvs.*;
import com.olf.openjvs.enums.*;

/**
 * Script Name: launchSharePointURL
 * Description: This script launches a specified Microsoft SharePoint URL in the default web browser.
 * Author: [Your Name]
 * Date: [Date]
 */

public class launchSharePointURL implements IScript {

    @Override
    public void execute(IContainerContext context) throws OException {
        // Define the long SharePoint URL
        String sharePointUrl = "https://your-long-sharepoint-url.com";

        // Log the initiation of URL launch
        OConsole.oprint("Initiating URL launch: " + sharePointUrl + "\n");

        // Launch the URL in the default web browser
        try {
            java.awt.Desktop.getDesktop().browse(new java.net.URI(sharePointUrl));
            OConsole.oprint("Successfully launched URL: " + sharePointUrl + "\n");
        } catch (Exception e) {
            // Log any errors that occur during the URL launch
            OConsole.oprint("Error launching URL: " + e.getMessage() + "\n");
            // Optionally, log the stack trace for detailed error analysis
            OConsole.oprint(e.toString() + "\n");
        }
    }
}

In the process of writing the code, it hit one compilation error that it was able to resolve with a further prompt containing the error message. The plugin worked and successfully launched the specified URL in the default browser. It should be noted that an alternative solution was used to resolve the underlying issue.

Other Uses

ChatGPT is also quite good at deciphering screenshots. Not sure what a query is doing? Take a screenshot and the AI will spit out a quick summary, as well as explain some complex ad hoc SQL some unscrupulous consultant has whacked in there. ChatGPT will also happily analyze the custom SQL for performance bottlenecks and suggest optimizations. If using this to document multiple queries, it will also successfully identify duplicates, even when that task was not in the initial request. A good use of this may be to document all Accounting Desktop queries to understand where there is duplication and potential for rule rationalization.

ChatGPT is also very good at analyzing error logs. When processes are failing there can be a lot of unnecessary data written to the logs or console. The AI will be far quicker and more efficient in reviewing everything and pulling out lines that may explain the failure.

“Boy, do I hate being right all the time.”

Despite the promising results of using ChatGPT for supporting Findur JVS plugin maintenance, there are several challenges and limitations, not to mention it always thinks it is right about everything. It is also worth mentioning that many of the changes proposed by ChatGPT would be easily identifiable/fixable to an experienced developer using Eclipse.

Context Limitations

Challenge: ChatGPT operates on a session-based interaction model, meaning it does not retain context between sessions. This requires repeated input of background information for each new query.
Mitigation: To address this, developers need to provide comprehensive context within each session and keep sessions focused on a single issue or enhancement to minimize the loss of context. Some versions of ChatGPT also allow custom instructions to retain context between sessions.

Complex Code Understanding

Challenge: Some JVS plugins involve highly complex and interdependent code structures. ChatGPT occasionally struggled to fully comprehend intricate dependencies and nuances within the code.
Mitigation: Developers can break down complex code into smaller, more manageable snippets when querying ChatGPT. Providing detailed explanations and context can help improve the accuracy of ChatGPT’s responses.

Documentation Consistency

Challenge: Ensuring consistency in documentation updates can be difficult, as ChatGPT’s responses may vary slightly in style and terminology.
Mitigation: A standard documentation template and guidelines can be established. Developers should review and standardize ChatGPT’s outputs to maintain consistency across all documentation.

Lack of Real-time Code Integration

Limitation: ChatGPT cannot directly interact with the Findur system or the JVS plugin repository, limiting its ability to perform real-time code analysis or updates.
Impact: Developers must manually copy and paste code snippets, which can be time-consuming and prone to human error. Direct access to a Findur database would make the tool much more powerful.

Dependency on User Input

Limitation: ChatGPT relies heavily on the input provided by users, meaning the quality and accuracy of its responses are directly influenced by the clarity and completeness of the queries. Sometimes it got confused and thought JVS was a form of Jython.
Impact: Clear, specific, and well-structured queries should be provided to ChatGPT. Including as much relevant detail as possible helps in obtaining precise and useful responses. The structure of your prompt is important.

Static Knowledge Base

Limitation: ChatGPT’s knowledge is based on pre-existing data and does not include the latest updates or specific organizational practices that have emerged after its last update. There is limited access to OpenJVS methods, and the model can struggle to replace those that are deprecated.
Impact: Developers need to supplement ChatGPT’s suggestions with up-to-date knowledge and organizational specifics.

Limited Troubleshooting Capability

Limitation: While ChatGPT can provide valuable insights and suggestions, its ability to diagnose and resolve highly complex or novel issues is limited.
Impact: For particularly challenging problems, developers may still rely on their expertise and in-depth analysis.

“Life Finds a Way”

There is clear, but limited, potential for using ChatGPT to assist in Findur JVS plugin maintenance. By providing detailed summaries, adding comments, and making minor code improvements, ChatGPT can be a valuable tool in streamlining certain aspects. Despite the challenges and limitations, ChatGPT can aid developers or expert users with at least some coding knowledge.

You need to go into any engagement with ChatGPT with your eyes open. It is an “expect the worst and hope for the best” type of scenario. ChatGPT will make mistakes; it tells you as much at the bottom of the webpage. What matters is whether you can identify those mistakes. If you can, there are aspects of Findur JVS plugin maintenance it can help you with.

This obviously does not mean the end of OpenJVS developers. ChatGPT cannot do anything in isolation and the person feeding the prompts needs to be able to understand enough to pose the right question and parse the output. That is something that Lucido can help you with, so please reach out if anything in this paper has piqued your interest.

What’s New

Capital Markets vs Energy: A Case for Modernization