Navigating Grey Zones – Consistently Picking the Right Tasks

In our previous discussion, we explored the versatility of the Solver API in solving complex decision-making problems. Now, let’s dive deeper into a specific scenario: prioritizing tasks within an organization by considering multiple dynamic factors. By harnessing the Solver API’s power, businesses can not only streamline task allocation but also adapt in real-time to changing business conditions, ensuring optimal resource utilization and client satisfaction.

The Multifaceted Nature of Task Prioritization

Task prioritization is no longer just about matching tasks with employee skills—it’s about balancing various dynamic business factors to maximize efficiency and client satisfaction. Let’s break down some of these aspects and explore how the Solver API can optimize task prioritization:

1. Client Satisfaction Level

Client satisfaction is paramount in any organization’s success. Tasks that directly contribute to improving client satisfaction should be prioritized accordingly. By considering client feedback, project milestones, and service level agreements (SLAs), organizations can ensure that tasks are aligned with client expectations.

2. Client Revenue Share

Not all clients or projects are created equal. Some clients may contribute more to the organization’s bottom line or strategic objectives. Tasks associated with high-value clients or projects should be given priority to maximize return on investment and foster long-term relationships.

3. Task Expected Revenue Generation

Tasks directly contributing to revenue generation are prioritized to optimize profitability and business growth.

5. Employee Cost

Balancing task prioritization with employee cost helps optimize resource allocation and minimize operational expenses.

6. Employee Skills and Proficiency

While financial and client considerations are great, if you can’t match the demand to the best resource, the overall objective will not be realized. Not only do you need to find employees with the right skills but you might also need to consider their proficiency for a particular skill demanded.

Leveraging the Solver API for Dynamic Prioritization

Here’s how the Solver API can be leveraged to address these dynamic aspects of task prioritization:

1. Real-time Data Integration

Integrate real-time data sources to dynamically update task priorities based on changing client satisfaction levels, client value, SLAs, revenue projections, employee costs, and utilization rates.

2. Dynamic Scoring Logic

Configure the Solver API to dynamically adjust scoring logic based on fluctuating business conditions, ensuring tasks are prioritized optimally to meet evolving requirements.

3. Continuous Optimization

Implement continuous optimization algorithms within the Solver API to adapt task priorities in response to real-time changes, maximizing efficiency and client satisfaction.

4. Scenario Planning

Utilize the Solver API for scenario planning, allowing businesses to simulate different task prioritization strategies and their impact on various metrics, enabling informed decision-making.

5. Adaptive Resource Allocation

Enable adaptive resource allocation capabilities within the Solver API to allocate resources efficiently based on task priorities, employee availability, and skillsets.

Sample Use Case

Let’s consider a scenario where a digital marketing agency, Digital Dynamics, needs to prioritize tasks for their employee, John Smith, based on various factors such as client satisfaction level, task revenue generation, and John’s skills and proficiency.

Here’s a overview of this simplified use case:

Tasks to Prioritize

Task	Client Satisfaction Level	Client Revenue Share	Task Revenue	Task Cost	English Proficiency Required	French Proficiency Required	Spanish Proficiency Required
Online Campaign Design	85	40	500	50	8	0	0
Content Creation	75	60	300	30	8	0	0
Translation Services	0	0	700	70	9	9	9

Task Prioritization Factors:

• Client Satisfaction Level %: Indicates the level of satisfaction of the client with the services provided.
• Client Revenue Share %: The percentage of revenue generated from the client’s business.
• Task Revenue: The revenue generated from completing the task.
• Task Cost: The cost associated with completing the task.
• French, English, Spanish Proficiency Level Required: The proficiency level required for each language, ranging from 0 to 10.

John’s Employee Profile

• English Proficiency: 8
• French Proficiency: 4
• Spanish Proficiency: 3

Tooling

In order to simplify the creation of this use case, I took advantage of the capabilities present in another tool I’m working on as a hobby project that leverages the Solver API in the background, www.dretza.com. While the Solver API doesn’t depend on Dretza in any way to be used, the tool allowed me to quickly put together this demo together by:

• Defining a new item type
  • To support Task as something that be considered in a problem to solve
• Defining new descriptor types
  • To support the characteristics of the tasks and also to define the problem constraints
  • Descriptor types can be reused across item types
• Defining new descriptors
  • While not used specifically in this demo, they are typically used to define valid options for a combination of item type and descriptor type
    • A descriptor of type brand for a car item type would be Ford while the same descriptor type for a television item type would be Sony
• Storing tasks and their characteristics
• Storing the problem constraints
  • A simple way to be able to reload the prioritization criteria
  • While the tasks to be prioritized can be added and remove, you can keep the same problem constraints and simply re-evaluate an updated list of tasks
  • You always have the option of making changes to the problem constraints by adding, removing or updating them

Dretza also provides additional capabilities that are beyond the scope of this discussion through its usage of generative AI for tasks such as data extraction, validation, summarization, Q&A etc.

Results: Bias on Revenue Generation

As you can see in the results above, the Solver API tries to balance budget and skill but with a special emphasis on revenue generation. You will also notice below a slight tweak that’s happening on Dretza as to what happens when a certain problem constraint is favorably exceeded (Bonuses). In this instance, the task was for a client which exceeded the requirement of being at least of a particular revenue share, so certain tasks were rewarded. (i.e. we wanted to favor bigger clients). You can also see how a particular task was penalized for the same constraint as that task might be for a new client with an unproven business flow.

Results: Bias On Cost

On the other hand, if we put a bigger emphasis on operational costs, we can see that cheaper tasks to accomplish now get favored. Also note how the Translation Services job gets severely penalized for multiple reasons (lack of skills, high cost, not targeting the right client profile).

Benefits of Dynamic Task Prioritization

Dynamic task prioritization powered by the Solver API offers several benefits:

• Enhanced Client Satisfaction: Prioritizing tasks based on client satisfaction levels ensures timely resolution of critical issues, leading to higher client satisfaction and retention rates.
• Optimized Resource Allocation: Balancing task priorities with employee costs and utilization rates helps optimize resource allocation, minimizing operational expenses while maximizing productivity.
• Improved Decision-Making: Real-time data integration and continuous optimization empower businesses to make informed decisions, adapting task priorities to changing business conditions and client needs.
• Increased Efficiency and Profitability: By prioritizing tasks with the highest client value and revenue generation potential, businesses can maximize profitability and achieve strategic objectives.

Conclusion

In conclusion, dynamic task prioritization is essential for modern businesses operating in fast-paced and ever-changing environments. By leveraging the Solver API’s capabilities and integrating real-time data sources, businesses can optimize task allocation, enhance client satisfaction, and drive profitability. As business conditions continue to evolve, the ability to adapt task priorities dynamically is crucial for maintaining competitiveness and achieving long-term success. With the Solver API, businesses can stay agile, responsive, and proactive in meeting the demands of today’s dynamic marketplace.

A New Era of LLM-Powered Negotiations

Introduction

In a world increasingly captivated by the potential of AI, Microsoft’s recent framework, AutoGen, stands out. AutoGen revolutionizes how we view and interact with large language models (LLMs). It’s not just an advancement; it’s a gateway to a new dimension where AI entities, known as agents, can communicate among themselves to solve complex tasks. The real kicker? These agents are customizable, conversable, and can incorporate human participation fluidly.

What is AutoGen?

Microsoft AutoGen is a robust framework that lays the foundation for developing applications powered by LLMs. These applications are not limited to single-agent responses but encompass multiple agents that can engage in dialogues to reach a consensus or solve problems. These agents are highly adaptable, able to mimic roles ranging from customer service representatives to negotiators representing different parties.

The Negotiation Example: Union vs. Government

To illustrate the power of AutoGen, let’s delve into an example where two LLM agents (think ChatGPT vs ChatGPT), one embodying a union and the other representing the government, engage in a negotiation. The scene is set with each agent bearing its own core objectives, mirroring real-world positions in a negotiation.

Context Setting

• The Union Agent: Programmed with the objective to secure a 20% salary increase over 5 years with adjustments for inflation.
• The Government Agent: Tasked with maintaining financial sustainability while addressing the union’s demands.

The Dialogue Begins

The conversation unfolds as each agent communicates through a chat manager, a central hub in the AutoGen framework that orchestrates the negotiation.

Union’s Opening Statement

The Union agent firmly insists on the 20% increase, justified by the members’ increased productivity and contributions to public services. It stresses that full inflation adjustments are non-negotiable to protect the workers’ purchasing power.

Government’s Proposition

In response, the Government agent acknowledges the union’s demands but suggests a balanced approach. It proposes a smaller fixed salary increase paired with performance-based bonuses to incentivize productivity. Inflation adjustments, according to the agent, should be partial and revisited annually to reflect economic realities.

Negotiation Progresses

The Union’s Flexibility

While standing its ground on the 20% increase, the Union shows willingness to negotiate, indicating the importance of clear and fair criteria for performance bonuses. It agrees to the idea of annual reviews of inflation adjustments, provided they are full and fair.

Government’s Reconciliation

Recognizing the Union’s concerns, the Government agrees that bonuses should not replace fair salary increases. It commits to revisiting inflation adjustments annually with transparency and maintaining the real value of employees’ wages.

A Step Towards Agreement

Both agents, through a series of exchanges, demonstrate a shared goal: to ensure the well-being of public service employees and the sustainable delivery of quality public services. They commit to transparency, open dialogue, and a balanced solution that respects the employees’ needs without compromising public finances.

Conclusion: The Role of AI in Human Affairs

The hypothetical conversation between the AutoGen agents encapsulates the immense potential of AI in facilitating complex human negotiations. It’s a testament to how conversational AI can model rational discourse, weigh diverse objectives, and explore mutually beneficial solutions.

While this scenario is a simplified representation, it hints at a future where AI could assist or even take part in real-world negotiations, ensuring efficiency and objectivity in discussions that impact societies and economies.

As Microsoft’s AutoGen continues to evolve, it may not be long before we witness its applications in various domains where decision-making is key. This could herald a new age of AI-mediated communication, one where the line between human and machine negotiation strategies becomes increasingly blurred, yet the outcomes become ever more focused on collective welfare and understanding.

As I’m continuing work on my personal project with Dretza, I have several ideas where I think multi-agent conversations could elevate the product.

Full Conversation Transcript

If you’re curious to see how that conversation went down in details, here’s the full transcript.

User_proxy (to chat_manager):

I need to have the union and the government to agree on the salary increase and benefits. The union wants a 20% increase over 5 years and inflation adjustments.

Government (to chat_manager):

As an AI, I suggest you adopt strategies that balance the needs of all parties involved to maintain financial sustainability. Develop a negotiation strategy that both respects the union’s demands and maintains manageable public expenses. You could potentially propose a smaller fixed salary increase over the years, supplemented with performance-based bonuses to incentivize productivity. For inflation adjustments, rather than a full adjustment, you could suggest a partial adjustment that is revisited annually. Negotiations should maintain an open dialogue, and it could be beneficial to involve an independent arbitrator if there is difficulty reaching an agreement. Expanding on financial education and the current economic constraints might also help employees understand why it’s necessary to restrict salary increases. Always remember to emphasize the mutual goal of quality public service delivery and the role everyone plays in achieving it. Open and transparent communication is key, as it builds trust and leads to mutual understanding.

Union (to chat_manager):

From a Union perspective, we should insist on the 20% increase since it represents fair compensation for the hard work and dedication of our members. It’s important to highlight that the value of our work has increased along with our productivity. We can potentially compromise on spreading the increase over a 5 years period or slightly more, but we should remain firm about the percentage increase. Regarding inflation, the Union should advocate for a full inflation adjustment to prevent the purchasing power of wages from falling. If it is revisited annually, this could be beneficial, but only if it guarantees the full adjustment. In this negotiation, we can highlight the quality of public services our members contribute to and the economic consequences of underpaid and thus potentially unmotivated employees. We should work together with the government to find a solution that does respect the demands of our members and does not undermine public finances simultaneously. Always keeping in mind that open and transparent communication can lead to better understanding and solutions that are satisfactory to all.

Government (to chat_manager):

As a representative of the government, our goal should be to balance the interests of all parties involved. While we acknowledge and appreciate the hard work and dedication of the union members, we also need to consider the overall economic constraints and the impact on public finances. We propose a lesser fixed salary increase over the years, supplemented with performance-based bonuses as a way of rewarding productivity and dedication. This ensures that those who contribute more will receive commensurately more. Regarding inflation, we propose a partial adjustment, revisited annually, to retain a level of balance as inflation fluctuates over time. This ensures that our strategy is flexible and adaptable in response to the economic climate. We understand that quality public services delivered by motivated employees are essential for the well-being of our society. Therefore, we should strive to find an agreement that respects the demands of the employees without compromising public finances. This includes maintaining open and transparent communication to ensure trust and mutual understanding. Together, we can maintain the quality of service our citizens deserve while ensuring the financial sustainability of our public services. —-

Union (to chat_manager):

As a representative of the Union, it’s our duty to ensure that our members receive fair compensation for their work. That said, we understand the need to balance this with economic sustainability. While we believe a 20% increase would be ideal, we’re willing to negotiate this. The idea of performance-based bonuses could potentially supplement base salaries, but we must take into account that these could generate inequalities and depend on factors out of the control of employees. We need to establish clear and fair criteria for these bonuses. In regards to the inflation adjustment, we believe it should be full to ensure the real value of salaries is not decreased over time. However, we agree to this being revisited annually as it can ensure adjustments are made according to real inflation rates. The quality of public services is totally linked to employee motivation, and this, to a large extent, comes from feeling valued and fairly compensated. Thus, we hope to work with you to reach an agreement that respects our members’ demands without compromising the financial health of public services. As always, we value open and transparent communication and hope through this, we can find a mutual understanding.

User_proxy (to chat_manager):

Government (to chat_manager): As the government’s representative, we appreciate the openness of the union to negotiate the 20% salary increase. We recognize the importance of fair compensation for the hard work your members do and are committed to balancing this with fiscal responsibility. Regarding performance-based bonuses, we will ensure that the criteria is fair, transparent, and fully communicated to everyone. It’s our intention that this method would motivate even better performance and reward those who go above and beyond their duties. We appreciate your understanding about the annual revisiting of the inflation adjustment. Given the unpredictable nature of the economy, revisiting the adjustment annually will ensure that it aligns with actual inflation rates. We completely agree with you that the quality of public services is closely linked with employee motivation. And in this spirit, we look forward to working with you to reach an agreement that respects your demands, provides fair compensation, and ensures sustainable public services for our population. We also value this open and transparent communication and believe, through genuine dialogue, we can strike a balance that benefits everyone.

Union (to chat_manager):

As the Union’s representative, we appreciate the government’s considerations for a fair compensation and for understanding the importance of adjusting salaries according to inflation. We believe we can find a middle ground on the proposed salary increase but maintaining the integrity of the deal to benefit our members will be our top priority. The introduction of performance bonuses should indeed be done in a transparent, fair and objective way in order to motivate our members further. However, we need to ensure that these bonuses do not become a substitute for necessary salary increases. They should be an additional reward, not the main component of income growth. As for the annual revisiting of the inflation adjustments, we agree that this method can be a good compromise, provided that any adjustments fully reflect the inflation rates and ensure that real wages don’t decrease. We are ready to cooperate and negotiate to reach a balanced agreement that recognizes our members’ contributions and ensures the sustainability of public services. Through this open dialogue, we can arrive at a mutually satisfactory solution. Let’s move forward with the welfare of our members—and by extension, the general public—at the heart of our negotiations.

Government (to chat_manager):

As the representative of the government, we hear the Union’s concerns and agree that any performance bonuses should not be a substitute for necessary salary increases. They should only serve as an additional reward for those who excel at their job duties. Specifics of such a bonus structure can be decided through further discussions. As for the inflation adjustments, we pledge to revisit the percentages annually with transparency ensuring they truly reflect the current economic climate and maintain the real value of our hard-working public employees’ salaries. We hold a shared vision of ensuring the welfare and motivation of our devoted public service employees. We believe that finding the balance between fair compensation and fiscal sustainability is key to the continued delivery and even improvement of public services. We look forward to further discussions, compromise, and transparency in our dialogues aimed at reaching a mutually acceptable agreement. We, too, place the welfare of our employees—and by extension, that of the general public—at the core of our negotiations.

Union (to chat_manager):

As the Union’s representative, we genuinely appreciate the government’s acknowledgement that performance bonuses should not replace necessary salary increases and should be an additional reward for our hardworking members. In working out the specifics of such a scheme, we aim to cooperate in all possible ways. Regarding the yearly inflation adjustments, we are pleased to see the commitment towards transparency and ensuring that these adjustments fully reflect the current economic scenario and prevent any reduction in the purchasing power of our members. It’s heartening to see that we share a vision of securing the welfare and motivation of our public service employees. We believe that the delicate balancing act between ensuring fair compensation and financial sustainability is crucial for the continued and improved delivery of public services to our citizens. We eagerly forward to further discussions and are committed to transparency and negotiation in order to reach best possible agreement. Always keeping the well-being of our members, and by default the general public, at the heart of our dealings.

When Retrieval Augmentation Generation Doesn’t Quite Cut It

One approach to answering user inquiries with LLMs and reference material is called Retrieval Augmentation Generation, RAG for short. While RAG can serve in a multitude of question/answering scenarios, it can fall short in certain situations.

A common challenge in conversation-based applications that leverages Large Language Models is constraining responses to a valid set of options that have been curated. This helps also addressing some of the problems caused by model hallucinations.

In order to overcome this in the personal project I’m working on, I had to come up with a way to make the output of the process more reliable and predictable.

Process Overview

Here’s an high level overview of the current version of the process. With each iteration of the product, this is getting fine tuned and made even more reliable.

Let’s break down the key steps in the process of using this API:

1. Split User Input into Separate Tasks

The conversation begins with the user input, which may contain multiple tasks or questions. The API first parses and separates these tasks to address them individually.

2. Generate User Task Embedding

For each task, the API generates a task embedding—a condensed representation of the user’s intent and context. This embedding is crucial for the subsequent steps.

3. Query Domain Option Types For Most Similar Embeddings

To ensure valid responses, the API queries domain-specific option types for the most similar embeddings to the user’s task. This step narrows down the potential responses to options within the relevant domain.

4. Query Domain Option Type Values For Most Similar Embeddings

Building on the previous step, the API further refines the options by querying domain-specific option type values for the most similar embeddings. This step ensures that the responses align with the specifics of the user’s inquiry.

5. Craft LLM Prompt using Top K results from previous queries

With the options and values identified, the API crafts a Language Model (LLM) prompt using the top results from the previous queries. This prompt serves as the basis for generating an answer to the original user inquiry. This essentially allows the model to have only the right information in hand to generate a relevant answer to the user input.

6. Execute LLM Prompt with Function Calling

To obtain a structured response in JSON format, the API executes the LLM prompt using function calling. This ensures that the response aligns with the predefined options and values.

7. Parse LLM Output

The API parses the output generated by the Language Model, extracting structured information in JSON format. Having structured data is important as this structured data can now be used more safely for other downstream tasks in the conversation process.

8. Respond to User Inquiry

Finally, armed with the structured response, the API provides a tailored and accurate reply to the user’s inquiry. This response is not only relevant but also constrained within the valid set of options, ensuring a seamless and satisfying user experience.

Sample Use Case: Chatbot Requirements Capture

Let’s explore a scenario involving a chatbot designed to capture user requirements such as the one I’ve been working on in my personal project, illustrating the versatility of this API. In this context, user requirements can take on different types, such as specifying the brand of an item, which may only apply to certain item categories.

Consider the following aspects:

1. Diverse Requirement Types: User requirements can vary widely, including attributes like the brand of an item. However, these requirements are not universal and may be relevant only to specific types of items. For instance, when a user expresses a brand-related requirement, it’s essential to recognize the context in which this requirement applies.
2. Constraining Valid Options: Once the chatbot understands the user’s intent, especially when it pertains to a requirement of a certain type like “brand,” it may need to further refine the options. For example, if the user specifies a brand requirement, the chatbot should constrain the valid options to align with that specific requirement. This could involve filtering values to only include what’s relevant to a particular category.
3. Adapting to User Input: User input can sometime be ambiguous. Sometimes, a user might state their requirement by only mentioning the brand name. In such cases, the chatbot needs to reverse its logic, starting by identifying the specific option value embedding to understand that the user is referring to a particular brand. This adaptability is crucial, especially when dealing with complex user inputs that may not align with the chatbot’s pre-trained knowledge. For example, if the user simply states “I like Ford”, you need to have some pre-existing context to determine they are talking about Ford as a car brand and not Tom Ford or the Ford modeling agency.

Here’s an example of what it looks like in action in the product I’m working on:

In summary, the chatbot, powered by this API, excels in handling nuanced user requirements, even when they involve specific types like brands. It adapts its responses based on user input, ensuring that the conversation remains coherent and responsive, regardless of how users express their needs. This dynamic approach enables the chatbot to effectively capture and validate user requirements, providing structured and precise information in return.

In Conclusion

The API we’ve developed represents a significant leap forward in the world of chatbot and conversation-based applications. By streamlining the process of constraining responses to valid options, it enables chatbots to provide more accurate and relevant information to users. Whether you’re building a customer support chatbot or a virtual assistant, this API can enhance the user experience by ensuring that responses are not only informative but also within the desired scope.

As the world of natural language processing continues to evolve, tools like this API empower developers to create more sophisticated and efficient conversational interfaces. With the ability to handle complex tasks and constraints, chatbots built using this technology are poised to deliver a new level of user satisfaction and efficiency.

If that’s something of interest and would like to discuss further how this new API can be used in your scenario, feel free to reach out to me.

Navigating Grey Zones In Decision-Making: Introducing the Solver API

In today’s complex and ever-changing world, making decisions is rarely straightforward. Often, the best solution is not perfect but requires careful balancing of various factors and compromises. This is where the Solver API comes into play, enabling you to score and find the optimal option from a myriad of possibilities while considering a wide range of requirements. In this blog post, we’ll explore the inner workings of this powerful tool and its versatile applications.

The Core Principle: Descriptors

At the heart of the Solver API’s functionality are descriptors. Descriptors are used to describe the key attributes of items under consideration. They are simple, consisting of a name and a value. Here’s an example of what a descriptor looks like:

{
   "type": "item.color",
   "value": "Red"
}

Descriptors provide the necessary foundation for the Solver API to understand and evaluate items.

Defining the Problem: Requirements

To define the problem to solve, you need to specify the requirements of an optimal solution. A requirement is composed of several components:

• Type: Aligns with how the possible options were described.
• Data Type: Specifies the data type of the requirement.
• Values: One or more desired or undesired values.
• Scoring Logic: Allows fine control over how the requirement is compared to the possible option attribute. It includes:
  • Comparison Logic (e.g., Greater Than, Less Than, One Of, None Of).
  • Method (Reward or Penalize).
  • Weight (an arbitrary penalty if the requirement is not met).
  • Constraint Hardness (controls the hardness of the penalty, such as soft or hard).
  • Value Range (provides value boundaries for penalty calculation).

Here’s an example of a requirement:

{
    "type": "item.attribute.performance.zero-to-100-kmph-duration-in-s",
    "values": ["3"],
    "valueType": "float",
    "scoringLogic": {
        "comparisonLogic": "GreaterThan",
        "method": "Penalize",
        "weight": 28,
        "constraintHardness": "hard",
        "valueRange": {
            "minimumValue": "1",
            "maximumValue": "20"
        }
    }
}

These requirements help the Solver API understand the criteria for an ideal solution.

Solving Complex Problems

Once you’ve defined both the problem and the possible items, you can set the Solver API to work. It explores various items, evaluating them against the defined requirements. Given the potential complexity of the problem space, the Solver API employs efficient techniques to navigate it. Brute-forcing through the problem is not a viable option.

The Outcome: Scoring Information

After exploring different possibilities, the API returns the best item/solution for the given requirements along with its score. Additionally, it can provide a list of other considered items and their respective scores. The scoring information offers valuable insights into why a particular option was chosen.

Here’s an example of the scoring information returned:

"scoreExplanation": {
    "overallHardScore": -6,
    "overallSoftScore": 0,
    "overallScorePercentage": 0.84210527,
    "requirementsScorePercentage": 0.78571427,
    "budgetScorePercentage": 1.0,
    "brokenRequirements": [
        {
            "type": "item.attribute.performance.zero-to-100-kmph-duration-in-s",
            "scoringLogic": {
                "method": "Penalize",
                "comparisonLogic": "GreaterThan",
                "weight": 28,
                "constraintHardness": "hard",
                "weightRelativePercentage": 1.0,
                "valueRange": {
                    "minimumValue": 1.0,
                    "maximumValue": 20.0
                }
            },
            "values": ["3"],
            "valueType": "float"
        }
    ]
}

In this example, the item incurred a hard penalty of 6 points for not meeting a specific requirement. You can also see the impact in the calculation for the requirements and overall score for the item.

Versatile Applications

The Solver API’s versatility extends to various use cases:

• Finding the right product based on requirements.

An example of how the solver API is used in a real-world product

• Identifying the best operational item considering constraints.
• Selecting the next project from a portfolio backlog.
• Matching individuals to jobs based on skills and availability.
• Determining the next best action in a given situation.
• Evaluating items for straight-through processing.
• Conducting risk assessments by identifying indicators and issues.
• Providing operational guidelines based on business conditions.

Crucially, the Solver API enables real-time problem-solving, allowing you to dynamically adjust requirements to adapt to changing conditions—a vital feature in the ever-evolving landscape of business and decision-making.

If this is something of interest to you and your organization, don’t hesitate to contact me.

In conclusion, the Solver API offers a powerful and flexible solution for navigating the complexities of decision-making, helping you find the best compromise among multiple requirements. Whether you’re optimizing products, operations, or human resources, this tool can guide you through the grey areas, delivering optimal solutions in real time.

Doing assertions on unstructured content

In today’s fast-paced world, dealing with unstructured text data can be a daunting task, especially when you need to extract valuable insights or validate specific information. Whether you’re managing product catalogs or verifying expense reports, ensuring data accuracy is essential. That’s where a custom-built API can come to the rescue. In this blog post, we’ll explore a novel API developed for handling unstructured text content, validating data using user-defined assertions, and handling multiple possible outcomes. We’ll delve into the use cases, development process, and the immense potential of this API.

The Problem

Imagine you’re managing a product catalog, and you need to ensure that each item is categorized correctly. This task becomes even more challenging when you’re dealing with vast amounts of unstructured text data. Additionally, in scenarios like expense report validation, you need to compare data points between different sources for accuracy. The need for an efficient solution led to the development of a custom API that can be used in different contexts.

The Solution: A Flexible Text Validation API

The custom-built Content Assertion API is a versatile tool designed to handle unstructured text content with ease. It empowers users to define assertions and specify various possible outcomes for each assertion. Let’s break down its key features:

1. Accepting Unstructured Text Content

The API accepts unstructured text content as input, making it suitable for a wide range of applications. Whether you’re working with product descriptions, expense reports, or any other textual data, this API has you covered.

2. User-Defined Assertions

Users can define their own assertions to validate specific information within the text. For instance, in the case of a product catalog, you can create assertions to verify if an item belongs to a particular category or exclude irrelevant items.

3. Multiple Possible Outcomes

One of the unique strengths of our API is its ability to handle multiple possible outcomes for each assertion. This flexibility allows you to account for various scenarios, ensuring comprehensive data validation.

Use Cases

Let’s explore two practical use cases that highlight the power and versatility of our Text Validation API:

1. Product Catalog Management

Imagine you’re tasked with ingesting a massive amount of data from product catalogs. Your goal is to categorize each item accurately, excluding irrelevant ones, such as replacement parts. With our API, you can define assertions to check if the product descriptions align with the assigned categories. Possible outcomes might include “Category Matched,” “Uncategorized,” or “Irrelevant,” depending on the validation results.

2. Expense Report Validation

In another scenario, you’re responsible for validating expenses in expense reports. You need to cross-reference data from credit card statements with what’s reported by individuals. Assertions can be created to compare key data points, such as transaction amounts and dates. Possible outcomes may include “Matched,” “Discrepancy,” or “Missing Data,” providing a clear picture of expense report accuracy.

3. Customer Feedback Analysis

A typical scenario that can be encountered in regular business operations is the analysis of customer feedback. Using the Content Assertion API, one can extract key information from the unstructured provided by the customer. This structured information can then be used to prioritize the feedback received.

Development Process

Building this Text Validation API involved several key steps:

1. Data Preprocessing: We started by cleaning and preprocessing the unstructured text data to ensure consistency and readability.
2. Assertion Definition: Users can define assertions using a user-friendly interface, specifying the text patterns or conditions to validate against.
3. Outcome Configuration: For each assertion, users can set up multiple possible outcomes, allowing for comprehensive data validation.
4. Machine Learning Integration: Our API leverages machine learning algorithms to analyze and validate text data efficiently.
5. Scalability: We ensured that the API can handle large volumes of data, making it suitable for enterprise-level applications.

Conclusion

In a world where data accuracy is paramount, our Text Validation API stands as a powerful solution for handling unstructured text content, defining custom assertions, and managing multiple possible outcomes. Whether you’re dealing with product catalogs or expense reports, this API can streamline your data validation processes and improve overall efficiency.

As you explore the potential applications of this API in your projects, remember that its flexibility and versatility make it a valuable addition to any data validation toolkit. With the ability to tailor assertions and outcomes to your specific needs, you’ll be better equipped to tackle the challenges of unstructured text data with confidence.

When you need more than layout specific extraction (part 2)

There you go! A bit more technical details about the extraction pipeline covered in the introductory article When you need more than layout specific extraction.

High Level Process

Extract Text Content and Layout

In order for extraction to be performed, you need to extract the text content if the source is an image. When doing so, it’s important to maintain the structural elements of the document as much as possible because they have meaning that.

Convert Into Intermediary Text Format

With this step, you take the result from the image conversion into text output and try to maintain the structural elements of the document and convert that into a text format that’s more dense/optimized for subsequent extraction steps.

Generate Extraction LLM Prompt

Depending on how you interact with the extraction API, I have step where the LLM extraction prompt is generated based on the item type. For example, if I need to extract data for a stick vacuum, I retrieve the list of all the fields that need to be extracted for that item type along specific extraction configuration on a per field level.

Determine Appropriate LLM Model

Once you have a better idea of the extraction job at hand, you can now make a better decision as to which LLM model is optimal for the extraction task at hand. It’s also an opportunity to optimize costs when that makes sense.

Generate Extraction Batches

Once you have an idea of the scale/complexity of the extraction task at hand and the targeted model, you can now proceed to the next step by generating the actual extraction batches. This allows you to deal the various subtleties of each LLM token limits.

Process Extraction Batches

With everything good to go, you can now iterate through the extraction batches and process their outputs. It’s at that step you can start structuring the extraction outputs.

Sanitize Extracted Values

Depending on the extraction task, you might need to further sanitize the extracted values. For example, you might want to ensure the extracted value is within a particular value domain. At a minimum, you would validate if the extracted value is of the particular data type. For example, if you need to extract a float value and the extraction incorrectly extracted a string, this would be your opportunity to deal with that.

Send Response

Now with the extracted data values in hand, you can now simply send the response payload back to the API caller.

Conclusion

Hopefully this helped you understand a little bit better how the extraction pipeline presented in the first part better. If you’re interested in using the extraction API methods, don’t hesitate to contact me in the comments section.

When you need more than layout specific extraction

Over the last few years, I had the opportunity to be involved in multiple projects revolving around extracting structured information out of unstructured or semi-structured content. I was able to experiment with different tools with each their own advantages and drawbacks.

At the beginning of 2023, I embarked on a new personal adventure with the development of a new web site. To say it’s a challenge is an understatement! I went through two major versions including a complete rewrite in Vue3 (which I didn’t know starting the project). I learned a lot along the way. It also allowed me to dive back into Python. In other areas of the project I had the opportunity to work extensively with generative AI for a wide variety of tasks such as contextualized question answering and user chatbot input treatment to only name a couple.

In today’s post I’d like to dive into one of the capability that was an absolute must have to achieve the mission of the site.

To give some context, my new web site requires the ingestion of a wide variety of unstructured content from a variety of sources. For instance, item specifications might be present in unstructured text, or specifications tables provided by the manufacturer and each source of content might differ in layout and information provided. For each type of item you need to handle, you have various extraction subtleties to account for. Sometimes the specifications are abbreviated, sometimes they are expressed in different units that need to be standardized.

To tackle this challenge I had to develop a custom extraction pipeline to allow for the following:

• Configurable extraction process on a per item type basis
  • The process can be fine tuned on a per field basis for a particular item type
    • For example, how you extract battery specifications for a smart phone is different than how you would extract them for a large battery used in a solar system installation
• Layout independent extraction
  • It doesn’t require specific text labels to be present nor does it expect the information at specific location in the source document
• Language agnostic
  • For example, the output of the extraction can be in English while the source content is in Italian
• Missing data handling
  • For when the information is not present or not extractable in the source

By using the extraction API, one can go from this:

to this:

Out of curiosity, I tested the same extraction pipeline but with document types not currently required by the web site. One of the examples I tested with was a flight boarding pass:

For which the extraction API returned:

As you can see in the response above, the extraction pipeline was able to adjust to certain quirks in the extraction request. For example, it successfully separated the first name and last name without requiring specific instructions. Same principle was applied to extracting the flight origin and destination without the airport IATA code.

One crucial point to note about this extraction pipeline is that it doesn’t require any particular training. The above example was achieved on the first attempt. So the process was the following:

• In the API request
  • Provide the fields you want to extract in the extraction API request
    • Name, data type, possible values when applicable
  • Provide the source content to extract
• Submit the API request and get the structured response

The extraction pipeline is able to adapt more easily to a wide variety of extraction tasks because it understands the semantics of both the extraction task to perform but also the meaning present in the source content.

If you have a use case where you think this document extraction API might be valuable, don’t hesitate to contact me in the comments section.

Azure API Management – Creating Azure Functions Slot Based Backends

If you use Azure Functions slots and want to use those as backends in Azure API Management, you will learn the hard way that you don’t have the nice import experience that you have with the production slot. In fact, there’s is no way in the Azure Portal to select a slot as the backend. This means that you basically have to do one of the following annoying things:

• Option 1
  • Deploy the app you want to import in the production slot (yikes) and then perform the import
  • Save the running config to the APIM Git repo
  • Pull the config
  • Change backend config file to point to the proper resource id->i.e. the /slots/<slot name>
  • Push back to the changes to the APIM Git repo
  • Deploy the config from the repo
• Option 2
  • If you use Azure Functions on a dynamic plan, don’t bother with slots. Create a function app representing your environment/slot. That way, it will be an importable production slot you can see in the Azure Portal Import API functionality.
• Option 3
  • Use Azure Resource Explorer to create the backend config
  • Import your OpenAPI specs manually
  • Done

I’ll cover in a bit more details Option 3.

1. Go to https://resources.azure.com
2. Navigate the resource tree to find the Azure API Management instance for which you want to add a backend
3. Expand to get to the backends tree node
4. From there, click Create

5. Specify the backend config like:
6. Depending on how the authentication to your backend API works, you may need to create a new Named Value in APIM to store the API key to use.
7. Click to create the new backend

{
  "id": "/subscriptions/<apim subscription>/resourceGroups/<apim resource group>/providers/Microsoft.ApiManagement/service/<apim name>/backends/<backend name>",
  "type": "Microsoft.ApiManagement/service/backends",
  "name": "<backend name>",
  "properties": {
    "title": null,
    "description": "<description of backend>",
    "url": "<Url to reach your backend api>",
    "protocol": "http",
    "resourceId": "https://management.azure.com/<full Azure resource URI of the backend>",
    "credentials": {
      "header": {
        "x-functions-key": [
          "{{<azure functions key to user to authenticate>}}"
        ]
      }
    }
  }
}

With the backend now created, you will need to add an inbout policy to your API to forward the traffic to this new backend. It would look something like the following:

<set-backend-service backend-id="<backend name created above>" />

 With this in place, you should now see the backend properly in the API designer view. You should also be able to click on the link present on the backend name to navigate to the Azure resource as you would expect.

Hopefully this was helpful to you! If you have any questions or comments, feel free to post them in the comments section.

Running Materialize on Windows Subsystem for Linux

For those not familiar with the company and its product, Materialize offers a data warehouse database engine optimized for streaming.

You can check out an intro video here from Frank McSherry, their Chief Scientist (and also former member of Microsoft Research who worked on Naiad):

You can also checkout their product page here.

In this post, I will explain how to run Materialize Community edition on Windows using Windows Subsystem for Linux.

Installing materialized

1. If not already enabled, make sure you have Windows Subsystem for Linux installed by going to Turn Windows features on or off

2. You may need to restart your computer after enabling it
3. Once that done, you can head over to the Microsoft Store to install your favorite Linux distro. In my case, I picked Ubuntu.

4. After the installation is complete, you now launch Ubuntu from the Start Menu
5. Once you have the Ubuntu terminal up and running, you can sudo to root using: sudo su –
6. To install Materialize, simply run the following command

curl -L https://downloads.mtrlz.dev/materialized-v0.1.1-x86_64-unknown-linux-gnu.tar.gz | tar -xzC /usr/local --strip-components=1

7. Once that’s completed, you should now be able to start materialized using:

/usr/local/bin/materialized

8. If everything went well, you should see the following:

Connecting to Materialize

In order to install the mzcli client, you will need to run the following in another Ubuntu terminal:

apt get update
apt install python3-pip
apt install unzip
apt install postgresql-server-dev-10
apt-get install python3-venv

curl -L https://github.com/MaterializeInc/mzcli/archive/mtrlz.zip > mtrlz.zip
unzip mtrlz.zip

pip3 install pipx

python3 -m pipx ensurepath
export PATH="$HOME/.local/bin:$PATH"

pipx install ./mzcli-mtrlz

If everything installed properly, you should be able to get the following text client:

You should now be all set to start streaming data to materialized! Here’s more information on how to start using Materialize:

Getting Started

If you would like to know more about Windows Subsystem for Linux, check out the following page:

Windows Subsystem for Linux Documentation

Happy streaming!

Using Azure SQL Database from Bubble

You may need at some point to create publicly facing application rapidly that relies on data stored in a SQL Server database. In this post, I’ll show you how to connect to Azure SQL Database from Bubble, a low code application development environment.

As the first step, you will need to add plugin to your Bubble application called SQL Database Connector.

Once that’s done, you can now configure the connection string as follow:

mssql://<user>:<password>@<sqlServerHostName>.database.windows.net:1433/<databaseName>?encrypt=true

One thing to be aware of is that if you are using firewall rules to control what can access your Azure SQL Database server, it might be tricky to enforce as Bubble uses dynamic outbound IP addresses while making the connection. So you might be whitelisting an IP one day and the other day another one would need to be whitelisted. When that happens you would see a message like the following while trying to connect or initializing a query:

SQL Database Connector issue: Cannot open server ‘<sqlServerHost’ requested by the login. Client with IP address ‘<Bubble IP>’ is not allowed to access the server. To enable access, use the Windows Azure Management Portal or run sp_set_firewall_rule on the master database to create a firewall rule for this IP address or address range. It may take up to five minutes for this change to take effect.

The current workarounds for that situation would be one of the following:

• Get a Bubble dedicated plan: This would provide you with a static IP
• Use an Azure Logic App, a custom web app or Azure Function to wrap the functionality needed to access that database as an http method that would be usable through the regular Bubble API Connector

The next step would be to define a new Query in the connector configuration like the following example:

Now that we have configured the basics, you will be able to use this as the data source for a repeating group or a state on your page. The type of content would be the name of the query and you will need to use the Get data from an external API option to find your query while defining the data source. If the query uses parameters, you will be able to specify to which bubble element or state you want that to be bound to. For example, you might have a drop down on your page that you want to use as the source value for that parameter. Once the repeating group is bound to your query, you can simply bind the elements within is by starting with Current Cell’s <query name> and then the list of fields present on your query will show up.

Also note that you can define queries to insert, update or delete data as well. Instead of those having a “Use as Data” setting on the query, those would be of type Use as Action instead. You would then be able to user those as part of workflows on your page that can be called by event or user actions like clicking a button for instance.

Happy low coding! 🙂