Sadly the amazon kindle version is not readable by their kindle web reader - tells me to download android or ios version of kindle :(

Related HN discussion - https://news.ycombinator.com/item?id=41831735 using Prolog as an intermediate target language for LLM output improves their 'reasoning' abilities.

An application I am developing for a customer needed to read constraints around clinical trials and essentially build a query from them. Constraints involve prior treatments, biomarkers, type of disease (cancers) etc.

Using just an LLM did not produce reliable queries, despite trying many many prompts, so being an old Prolog hacker I wondered if using it might impose more 'logic' on the LLM. So we precede the textual description of the constraints with the following prompt:

-------------

Now consider the following Prolog predicates:

biomarker(Name, Status) where Status will be one of the following integers -

Wildtype = 0 Mutated = 1 Methylated = 2 Unmethylated = 3 Amplified = 4 Deleted = 5 Positive = 6 Negative = 7

tumor(Name, Status) where Status will be one of the following integers if know else left unbound -

Newly diagnosed = 1 Recurrence = 2 Metastasized = 3 Progression = 4

chemo(Name)

surgery(Name) Where Name may be an unbound variable

other_treatment(Name)

radiation(Name) Where Name may be an unbound variable

Assume you are given predicate atMost(T, N) where T is a compound term and N is an integer. It will return true if the number of 'occurences' of T is less than or equal N else it will fail.

Assume you are given a predicate atLeastOneOf(L) where L is a list of compound terms. It will succeed if at least one of the compound terms, when executed as a predicate returns true.

Assume you are given a predicate age(Min, Max) which will return true if the patient's age is in between Min and Max.

Assume you have a predicate not(T) which returns true if predicate T evaluates false and vice versa. i.e. rather than '\\+ A' use not(A).

Do not implement the above helper functions.

VERY IMPORTANT: Use 'atLeastOneOf()' whenever you would otherwise use ';' to represent 'OR'. i.e. rather than 'A ; B' use atLeastOneOf([A, B]).

EXAMPLE INPUT: Patient must have recurrent GBM, methylated MGMT and wildtype EGFR. Patient must not have mutated KRAS.

EXAMPLE OUTPUT: tumor('gbm', 2), biomarker('MGMT', 2), biomarker('EGFR', 0), not(biomarker('KRAS', 1))

------------------

The Prolog predicates, when evaluated generate the required underlying query (of course the Prolog is itself a form of query).

Anyway - the upshot was a vast improvement in the accuracy of the generated query (I've yet to see a bad one). Somewhere in its bowels, being told to generate Prolog 'focused' the LLM. Perhaps LLMs are happier with declarative languages rather than imperative ones (I know I am :) ).

I find that having GPT-4 write SQL queries to query the data source as needed to solve a complex task step-by-step also works pretty well (and you can give it the schema in form of CREATE TABLE). It's not exactly good at writing fast queries, but it can do some hella complex ones with nesting and joins to get exactly what it needs in one go.

I agree; we can do much more with LLMs if we focus on declarative-style languages.

This sounds like a very important observation.

A lightweight but very scalable, robust (like Erlang's 'let it crash') and distributeable complete Actor system for Java 21 and above. Collection of implemented Actor idioms supporting the 'Dust way'. Also see dust-ville ... a toy 'digitally twinned' town built as a tiny Dust example - complete with a murmuration of Starlings circling the power station :)

I recently has a consulting gig (medical informatics) that required English Declarative -> Imperative code. Direct code generation by the LLM turned out to be buggy so I added an intermediate DSL implemented in Prolog! The prompt described the Prolog predicates it had to work with and their semantics and the declarative goal. The resulting (highly accurate and bug free) Prolog code was then executed to generate the conventional (Groovy) imperative code that was then executed dynamically. In some hand-wavy way the logical constraints of using Prolog as an intermediate DSL seemed to keep the LLM on the straight and narrow.

This is great! I had been thinking of how to constrain and make the prompts precise so "modes of interpretation" of my prompt by the LLM could be limited. Something like how one would simplify one's language with a child to get it to comprehend and act within bounds.

Would you mind sharing more details of your approach and setup?

Sure, the app selects matching patients based on demographics, disease, prior treatments and biomarkers. It also has to be able to express numeric constraints as well ('no more than two surgeries ..', 'at least one of the following biomarkers'). The following prompt sets up the Prolog toolkit the LLM is allowed to use. The generated Prolog conjunction is then run through a Prolog meta-interpreter that generates the matching code. Though it sounds long-winded it is more than fast enough to generate responses to user queries in an acceptable time:

-----------------

  Now consider the following Prolog predicates:
  
    biomarker(Name, Status) where Status will be one of the following integers -
   
     Wildtype = 0
     Mutated = 1
     Methylated = 2
     Unmethylated = 3
     Amplified = 4
     Deleted = 5
     Positive = 6
     Negative = 7
   
    tumor(Name, Status) where Status will be one of the following integers if know else left unbound -
    
     Newly diagnosed = 1
     Recurrence = 2
     Metastasized = 3
     Progression = 4
    
    chemo(Name)
   
    surgery(Name)  Where Name may be an unbound variable

    other_treatment(Name)

    radiation(Name) Where Name may be an unbound variable
     
    Assume you are given predicate  atMost(T, N) where T is a compound term and N is an integer. 
    It will return true if the number of 'occurences' of T is less than or equal N else it will fail. 
   
    Assume you are given a predicate atLeastOneOf(L) where L is a list of compound terms. 
    It will succeed if at least one of the compound terms, when executed as a predicate returns true.
   
    Assume you are given a predicate age(Min, Max) which will return true if the patient's age is in between Min and Max.
     
    Assume you have a predicate not(T) which returns true if predicate T evaluates false and vice versa. 
    i.e. rather than '\\+ A' use not(A).
   
   Do not implement the above helper functions.
  
   VERY IMPORTANT: Use 'atLeastOneOf()' whenever you would otherwise use ';' to represent 'OR'.  
   i.e. rather than 'A ; B' use atLeastOneOf([A, B]).
    
  EXAMPLE INPUT: 
   Patient must have recurrent GBM, methylated MGMT and wildtype EGFR. Patient must not have mutated KRAS.  
  
  EXAMPLE OUTPUT:
    tumor('gbm', 2),
    biomarker('MGMT', 2),
    biomarker('EGFR', 0),
    not(biomarker('KRAS', 1))
  
  Express the following constraints as a Prolog conjunction. 
        Do not enclose the code in a code block. Return only the Prolog code - no commentary.
        Be careful to use only the supplied constraints, do not add any: 
        
        $constraint

Very Neat! Appreciate your sharing this.

Just to be clear, your "EXAMPLE OUTPUT" is what is then fed to your prolog meta-interpreter to generate executable code in some other language (you mentioned Groovy) which is actually run i.e. answers the user query. Essentially then, a context is bounded by "pidgin Prolog" (i.e. Prolog+Natural Language) for the LLM and then user queries in Natural Language are submitted against it to generate valid Prolog code. This can be thought of as the logic/constraints of Prolog inference engine in the input modulating the interpretation/inference of the accompanying natural language by the LLM to keep it "on the straight and narrow" towards an accurate output.

I was actually thinking of using "Structured English" (https://en.wikipedia.org/wiki/Structured_English) for this and maybe build a CASE Tool using LLMs for round-trip software engineering.

    tumor('gbm', 2),
    biomarker('MGMT', 2),
    biomarker('EGFR', 0),
    not(biomarker('KRAS', 1))

Note well: this is not valid Prolog code. If you put it in a file and consult it with a Prolog interpreter you'll get multiple errors.

You could call it Prolog pseudocode but in that case you don't need Prolog-like notation. You can just state your "constraints" in natural language. Have you tried this?