Compiling Higher-Order Specifications to SMT Solvers: How to Deal with Rejection Constructively

Modern verification tools frequently rely on compiling high-level specifications to SMT queries. However, the high-level specification language is usually more expressive than the available solvers and therefore some syntactically valid specifications must be rejected by the tool. In such cases, the challenge is to provide a comprehensible error message to the user that relates the original syntactic form of the specification to the semantic reason it has been rejected. In this paper we demonstrate how this analysis may be performed by combining a standard unification-based type-checker with type classes and automatic generalisation. Concretely, type-checking is used as a constructive procedure for under-approximating whether a given specification lies in the subset of problems supported by the solver. Any resulting proof of rejection can be transformed into a detailed explanation to the user. The approach is compositional and does not require the user to add extra typing annotations to their program. We subsequently describe how the type system may be leveraged to provide a sound and complete compilation procedure from suitably typed expressions to SMT queries, which we have verified in Agda.


Introduction
As the performance of SMT solvers and other automatic theorem provers has improved, they have been applied to a wide range of domains, including policy verification [2], program synthesis [24,25], model-based testing [16] and neural network verification [14].However, manually writing queries for these solvers is tedious and error-prone, so tools often provide a higher-level domain-specific language (DSL) which is then compiled down to an equisatisfiable set of queries.
Such DSLs aim to provide a layer of abstraction between the user and the solvers, but do not entirely succeed.The main failure point of the abstraction is that each individual solver only supports a limited class of problems (known as "logics" in SMTLib) and that some classes of problems are more difficult than others.Therefore, an apparently innocuous tweak to a high-level specification may change the class of problems the compiled queries belong to and consequently the choice of solver.In turn, this may drastically increase the time taken to check the specification or even render it unable to be solved if no suitable solver is available.
We have encountered the latter issue while designing a DSL for writing specifications for neural networks solvers.Due to their large size and inherent complexity, neural networks require specialised solvers [14,29] which generally only support problems that belong to some subset of quantifierfree linear real arithmetic (QF_LRA).As we made our language more expressive, it became increasingly easy to write specifications that cannot be compiled to an equivalent set of queries in QF_LRA.
To illustrate how a user may write such a specification, consider a higher-order functional DSL for writing specifications equipped with the quantifiers forall and exists as first-class constructs in the language.We would like to compile specifications about some function  : R  → R down to queries for a QF_LRA solver.The exact semantics of  is unimportant; it could be an existing piece of C++ code, a neural network or an uninterpreted function to find a solution for.The user begins by writing a function that asserts some value  is in the range of the function  : inRange : Real -> Bool inRange y = exists .f  ==  This can then be reused to modularly specify that, for example, 0 and 1 are in the range of  : zeroAndOneInRange : Bool zeroAndOneInRange = inRange 0 and inRange 1 This specification is equisatisfiable with the quantifier-free query '  == 0 ∧   == 1' and which can then be solved by a QF_LRA solver that can handle  appropriately.However, the user may then use the innocuous inRange definition to specify that  is surjective: surjective : Bool surjective = forall .inRange  Fully expanded, this specification has alternating forall and exists quantifiers and therefore there is no equisatisfiable quantifier-free set of queries it can be compiled to.

Existing Approaches
One approach to addressing this issue is to design the syntax of the DSL so that all expressions in the language belong to the supported class of problems.For example in Liquid Haskell [27] and Cryptol [17], exists is not a first class construct in the language and therefore it is syntactically impossible for users to write non-quantifier-free statements.In particular, it would not be possible to define inRange in the examples above.While restricting quantifiers in this way is feasible, applying the same approach to non-linear expressions is significantly more difficult as multiplication is almost universally a first-class construct in languages.The approach also inhibits modularity, as functions containing quantifiers (e.g.inRange) cannot be declared and then reused in multiple places, (e.g. as in zeroOneInRange).Finally, it is inherently inapplicable when the DSL is compiled to multiple backends with different capabilities, e.g. a solver and some other non-solver, or multiple solvers supporting different logics.
An alternative approach is to first normalise the specification to remove all functions and then subsequently perform a simple syntactic check.This approach is adopted by the Z3 SMTLib frontend [11].While this procedure is both sound and complete, normalisation discards information about the specification the user has actually written.Therefore when the syntactic check subsequently encounters a problem, it cannot explain why or where in the source file a problem has occurred.For example, asking Z3 to solve an SMTLib encoding of surjective with a QF_LRA solver results in an error saying that the quantifiers are unsupported but cannot provide useful information to the user about the location of the quantifiers or how they interact.Although in this case it is easy to spot the problem, it may not be so easy in a large real-world specification where the definition of inRange may be far from surjective or used through a chain of intermediate function calls.
Finally, tools such as F* [26] throw generic error messages that do not even mention alternating quantifiers.We have collected example messages in Appendix A.

Our Contributions
In this paper, we propose a solution to this problem in the form of a type-system that can be used to soundly approximate membership of QF_LRA for an expressive specification language with higher-order functions.The basic idea is for the compiler to internally refine the Bool and Real types into families of types indexed by quantities tracking precisely how they use quantified variables.For example, one type might be the type of Booleans which universally quantify over real numbers which are used linearly.Polymorphism over these quantities is retained via a standard higher-order unification-based type-checker and what we believe to be a novel combination of Haskell-style type-class propagation [28] and Idris-style implicit generalisation [7].Crucially our approach does not require the user to add additional typing annotations, nor for them to understand advanced type system features.
Type-checking can then be used as a constructive decision procedure for membership, from which a proof of whether a specification can or cannot be compiled can be extracted.In the latter case, the proof can then be used to construct a detailed error message for the user.For example, passing in the example specification surjective results in the error: Cannot verify specifications with alternating quantifiers.In particular: 1. the inner quantifier is the 'exists' located at line 2, columns 12-18 2. which is returned as the output of the call to the function 'inRange' at line 1, columns 24-31 3. which alternates with the outer 'forall' quantifier at line 7, columns 13-19.
When the type-checking procedure succeeds, we are guaranteed that the specification can be compiled to a format suitable for a QF_LRA solver.In Section 4, we constructively demonstrate this fact by presenting a Normalisation by Evaluation (NbE) procedure [5] that accepts the fully elaborated output of our type checker and generates a semantically equivalent formula.NbE for our specification language is non-trivial due to the fact that our specification language allows mixing of if-then-else and uninterpreted function applications anywhere in terms.Both of these need to be extracted from the context in which they appear and hoisted to the level of Boolean constraints.We explain our method for accomplishing this lifting via a monad in Section 4.3.3.The totality, type-and semantic-preservation properties of our NbE procedure have all been formalised in the Agda proof assistant [21].
Our approach has been incorporated into the DSL of Vehicle [10], a new tool for writing and checking neural network specifications.While the paper focuses on QF_LRA, the technique is modular in the sense that it decides membership of quantifier-freeness separately from linearity.Therefore, while quantifier-freeness and linearity are arguably the two most important subsets of SMT logics, we hypothesise that the technique is more generally applicable to approximating membership of other SMT logics, e.g.integer difference logics [15].
The paper is laid out as follows.Section 2 presents an example of a higher-order DSL for writing specifications about functions over vectors.We then discuss the different classes of expressions tracked by our analysis.Section 3 describes our type based analysis procedure, and discusses its advantages and disadvantages compared to other approaches.Section 4 describes our formally verified normalisation procedure to SMT queries.

User Language
Figure 1 shows the grammar of a high-level language for writing specifications about abstract functions.A specification is a sequence of declarations of which there are three types: type synonyms, abstract functions declarations and definitions.Expressions contain many of the usual constructs available in functional languages, as well as sized vector literals with type-safe indexing and first-class universal and existential quantifiers.Due to space limitations it does not include redundant operations such as let-bindings and disjunction.Example 2.1 shows how the specification "function  is monotonically increasing in its second input" can be written in the language.where Vec   is the type of vectors of length  containing elements of type  and Index  is the type of indices into a vector of length .
A property is defined to be any definition whose type is Bool, and the goal of the compiler is to compile properties down to a set of queries for a QF_LRA solver.For example, the monotonic property might be compiled to the following satisfaction query: where all the variables are implicitly existentially quantified over.The 0 . . .4 and 0 . . .4 variables represent the elements of the original quantified vectors  and  respectively.This query is satisfiable if and only if the original monotonic property is false.
The language is equipped with a standard semantics that we will describe in Section 4. Here, it suffices to note two key points.Firstly, the language is designed for writing specifications rather than performing general-purpose computation and therefore it does not support recursion.Secondly, the semantics of the declared functions  are external to the specification.In the paper syntax will be written using a type-writer font, e.g.x == 2, and semantics using mathematical type-setting, e.g. = 2.
The aim of this paper is to analyse whether a specification is compatible with a quantifier-free real linear arithmetic solver (QF_LRA).This is a property of a specification's semantics rather than syntax, therefore we now classify expressions according to their semantics.For the linearity analysis, there are three classes of expressions: 1. Constant () -the expression's semantics only involves constants, e.g.
In the quantifier analysis we identify five classes, which we will call polarity classes: 1. Unquantified ( ) -no quantifiers in the expression's semantics, e.g.
The grammar for a high-level user language for writing specifications.

5.
Alternating quantifiers () -both universal and existential quantifiers present in the expression's semantics with a non-empty intersection of their scopes, e.g.

∀𝑦. ∃𝑥 . 𝑓 (𝑥) = 𝑦
Properties whose expressions are in the  , ∃, ∀ and  classes may all be checked with a QF_LRA solver.Expressions in ∃ are equisatisfiable with a QF_LRA expression.Expressions in ∀ may be negated to obtain an expression in ∃, and then the desired result is the negation of the output of the solver.Expressions in  may be split into multiple disjoint queries each of which are either in ∃ or ∀ and may be checked individually.However, detecting which class a given specification belongs to is not straightforward.For example, negating a Boolean expression will flip the polarity of the quantifiers within the semantics.Therefore although the following only contains forall quantifiers syntactically, semantically it has alternating quantifiers and therefore is a member of the  class: forall .not forall .  !=  Furthermore, the analysis needs to distinguish between quantifiers over variables with finite and infinite domains.For example, despite: forall .exists (i : Index 5).  ! <= 1 containing both a forall and an exists syntactically, it belongs to the ∀ class rather than the  class because the exists quantifies over the Index 5 type and therefore can be expanded out into five disjunctions.
In general, even for linearity, a syntactic check can never be sufficient as shown by the following example function: lam (x : Real).*  >= 0 whose output is constant if the input is constant and otherwise is non-linear.

QF_LRA Analysis
To facilitate the analysis, we now describe an intermediate representation (IR) for the language just defined and an associated type system.The latter is capable of soundly underapproximating membership of QF_LRA, and in the case of a negative result constructing a proof of non-membership which can be turned into readable diagnostic information for the user.

Intermediate Representation
The grammar for the IR is shown in Figure 2. At the expression level, it has exactly the same expressive power as the high-level language in Section 2.
However, at the type-level it is significantly more complex.As briefly discussed in Section 1.2, our proposed analysis tracks the linearity and polarity of expressions by turning the Real and Bool into type families indexed by the linearity classes (, ,  ) and polarities classes ( , ∀, ∃, , ) identified in Section 2. The Real type is indexed by the linearity annotations and the Bool type is indexed by both linearity annotations and polarity annotations.For example Real  is the type of real expressions that are linear in its free variables, and Bool ∀  is the type of Boolean expression that contain only forall quantifiers, at least one of whose quantified variable is used non-linearly.
This naturally leads to a type system that is a variant of System F1 [13,22], where types themselves have types, which are referred to as kinds.There are four base Kinds in our system: Type, Linearity, Polarity and Size, and new Kinds can be constructed from these using the function arrow.For example, the Bool type has kind:

Linearity -> Polarity -> Type
Types can also contain type-level variables.A new type-level variable  which has kind  can be abstracted over using the syntax '∀{ :  }....', commonly known as a pi-binder.As discussed in Section 1, one of our aims is to avoid the user having to write additional typing information.Instead missing types will be represented as meta-variables and written as ?where  is a unique identifying number.Metavariables are inserted by the elaborator and type-checker, and may represent type variables of any Kind.
In addition to the standard function type,  1 ->  2 , the IR also has the instance function type, {{ 1 }} ->  2 .This type enforces that the constraint  1 needs to be resolved in order to obtain something of type  2 .There are three classes of constraints in our system.The first is a set of type-classes that allow operators in the user language to be overloaded for multiple different types.For example, a HasEq constraint will be generated whenever an equality comparison operator is encountered, to enforce that the arguments are of a comparable type.The second and third groups of constraints exist to encode the relationships between linearity and polarity meta-variables.These will be generated by expressions in the user language that alter the polarity or linearity of an expression (e.g. a MulLin constraint is generated by a multiplication) or an event we would like to appear in the error messages (e.g. a InputLin constraint is generated when something is used as an input to a user-defined function).

Elaboration
Elaboration from the user language to the intermediate language is straightforward.Expression binders with a missing type get assigned a fresh meta-variable for their type.The builtin infix operations (+, *, if etc.) are turned into prefix operators.The quantifiers are changed from binding variables directly to builtins that take a lambda as their only argument, e.g.forall . is elaborated to forall (lam  .).Types are elaborated directly, and, with the exception of meta-variables, none of the additional type-level constructs introduced in the IR get added to the type signatures during elaboration.

Type Checking
Type-checking of a specification proceeds on a declarationby-declaration basis.For each declaration: 1. Fresh meta-variables for missing linearity and polarity type indices are inserted.2. A standard bidirectional type checking/inference pass is made over the declaration type and body, generating new meta-variables and constraints.3.An attempt is made to solve the meta-variables and constraints generated in the previous step.4. Constraints that track the application of user functions are introduced to the set of unsolved constraints.5.The declaration is generalised over by prepending unsolved meta-variables and constraints to the declaration's type signature.We will now describe these steps in more detail.

Inserting Linearity and Polarity Meta-variables.
By design, the user is unaware of the linearity and polarity annotations.Consequently after elaboration, the user's program will be ill-typed.Therefore the first phase of typechecking is to traverse all the types in the current declaration, inserting meta-variables into the indexed types.The two main indexed types are Bool and Real, however userdefined type synonyms that make use of Bool and Real will also be indexed by polarity and linearity annotations.
For example, consider the first two lines of the monotonicity specification in Example 2.1: Assume that the type-synonym Input has already been typechecked and generalised over to obtain: Input : ∀{ : Linearity}.Type Input = tlam { : Linearity}.Vec (Real ) 5 and that we are type-checking the declaration fun f.Then inserting fresh meta-variables ?0 and ?1 in  for the missing linearity types results in: fun  : Input ?0 -> Real ?1

Bidirectional Type Checking.
A second pass is then made over the declaration, using a bidirectional typechecking algorithm [9].We follow the lead of Agda [21] and Idris [7] by adapting it to insert a fresh meta-variable wherever a pi-binder is encountered, and wherever we find an instance argument we add the constraint contained within it to the set of constraints.The pi-binders can therefore be thought of as the mechanism for introducing new unknown types and the constraints encode the relationship between them, and hence provide information for how they should be solved.Note that one of the strengths of our analysis is that only the inference typing rules for the builtins and literals are non-standard and the core bidirectional algorithm remains unaltered.Correspondingly, we don't reproduce the full algorithm here.
Figure 3 shows the inference typing rules for the type and expression builtins.Real and Bool are now indexed families of types.The Vec and Index types are typed as expected.
Every builtin operation, with the exception of the lookup operation, has a constraint associated with it, and can be divided into two categories.The first are builtins that don't need to be overloaded and therefore the polarity and linearity constraints appear directly in their type.For example, + takes two Reals of linearity  1 and  2 and returns a third Real with linearity  3 , assuming that the constraint MaxLin  1  2  3 is satisfied (i.e. 3 is the maximum of  1 and  2 ).The second are builtins that are overloaded, in which case their typeclass constraints will first resolve the type, and only then add linearity and polarity constraints if required (see Section 3.3.3).For example forall generates the HasForall type-class constraint.
The typing of literals are shown in Figure 4. Real number literals are given the constant linearity annotation (), and Boolean literals are given constant linearity () and unquantified polarity ( ) annotations.Index literals require a constraint to enforce that its value is smaller than the size of the resulting type.The most interesting rule case is vector literals.Although the types of the elements of a vector need to be homogeneous in the high-level language, in the IR they may be heterogeneous.For example, the first component may be a Real , and the second a Real .This problem is solved by encoding an n-ary subtyping relation via the Subtypes constraint.

Constraint
Solving.The next step is to try to solve the set of constraints generated in the bidirectional phase.Each constraint is tried in turn, and is either a) returned to the set of constraints if it is currently blocked by an unsolved meta variable, b) removed from the set of constraints if it can be solved, the process of which may generate solutions for meta-variables and new (smaller) constraints, or c) marked as failed if it is provably unsolvable (in which case typechecking itself will fail).There are three different types of constraints that can be generated.
Unification constraints.Unification constraints are of the form  1 ∼  2 and assert that types  1 and  2 are equal.They can be generated both by the bidirectional pass and when solving type-class and polarity/linearity constraints.They are solved using a standard pattern unification algorithm, which may generate further unification constraints involving strictly syntactically smaller terms than the original two terms.For more details of the algorithm see the presentation by Nipkow [20].
Type-class constraints.Type-class constraints are used to resolve overloaded operators.Each type-class constraint Bool : Linearity -> Polarity -> Type Real : Linearity -> Type Vec : Type -> Size -> Type Index : Size -> Type !: ∀{}.Vec   -> Index  ->  (index)   Informally, the first rule hasForallReal can be read as follows: when quantifying over a Real with linearity  1 then i) unify  1 with  as any quantified variable is linear with respect to itself, ii) generate one new linearity meta-variable  2 and two new polarity meta-variables  1 ,  2 , iii) unify both the output type of the function and the output type of the quantifier with Bool types of linearity  2 and polarities  1 and  2 , thereby ensuring that linearity is preserved iv) constrain  1 and  2 to be related by the ForallPol constraint.
Derived as the least upper bound (⊔) operator from the lattice to the right.
Derived as the least upper bound (⊔) operator from the lattice to the right. (Unquantified) . Solutions for linearity and polarity constraints which alter their arguments.
Note that the use of type-class constraints gives us the flexibility to distinguish between finite and infinite quantifiers, as discussed at the end of Section 2. In particular, when solving the same constraint for the Index type the polarity is preserved without change.The rule for Vec ensures that this behaviour is inherited correctly depending on the vector's element type.The full set of rules for solving type-class constraints can be found in Appendix B.
Linearity/polarity constraints.The third type of constraints are those that operate directly on the linearity and polarity types.Polarity and linearity types flow upwards through the AST, from the constants and variables at its leaves, through the nodes of interest.Unlike type-class constraints, linearity and polarity constraints encode total functions, i.e. the last type argument in each constraint is deterministically derivable from the previous type arguments.Therefore they can never fail, although they may still temporarily block on unsolved meta-variables.
The solutions for the linearity and polarity constraints which alter their arguments are shown in Figure 6.For example if expression  1 has polarity ∃ and  2 has polarity ∀ then ' 1 and  2 ' would generate the MaxPol constraint, and would therefore be assigned polarity .Implication is an interesting case, as ImpliesPol can be defined in terms of NegPol and MaxPol.However, we keep ImpliesPol as a separate constraint so that we can distinguish between a quantifier that was negated by a 'not' and one that was negated by being on the LHS of an '=>' in the error messages.6.Their purpose is purely to track calls to user definitions so that the calls will show up in the error messages (e.g.line 2. in the example error message in Section 1.2).They therefore act as the identity function on linearity and polarity types as they have been presented so far.
How do they work then?Our approach, as presented so far, indexes every Bool with a linearity and polarity type.However, in Section 1.2 we promised constructive proofs.Therefore, in the same way that a compiler annotates nodes in the abstract syntax tree of an expression with their provenance in the source file, we annotate the linearity and polarity types with a proof of how that value was constructed.The grammar for these proofs terms is shown in Figure 7.For example, the non-linear type  has two pieces of provenance information attached, one each for the left and right sides of the non-linear multiplication respectively.
The constraints added during the bidirectional pass inherit the provenance of the operation that generated them.Therefore when solving a constraint, the name and provenance of the symbol that generated the constraint is available to be added to the constructed polarity or linearity type.When an InputPol/OutputPol/InputLin /OutputLin constraint is solved it leaves the actual linearity or polarity type untouched but appends a FunctionInput/FunctionOutput node to the provenance information associated with it.
The remaining issue is how to ensure these constraints are generated in the right place.In particular they need to be generated when the user function is applied, not when it is defined.The solution is to have them inserted into the ⟨prov⟩ ::= location in source file . Grammar for the internal proof structure of the linearity and polarity constraints.This is a refinement of the ⟨linearity ⟩ and ⟨polarity ⟩ classes in Figure 2.
type signature of user definitions after constraint solving.Concretely, we again iterate through all the unique polarity and linearity values in the input and output types of the type signature.For every value, we replace it with a fresh meta-variable and then link the old value and the new metavariable by the appropriate constraint.For example, suppose we have the following type after constraint solving: then after this operation we will have: : Real ?0 -> Bool ?1 ?2 and the following additional three constraints: InputLin   ?0 OutputLin   ?1 OutputPol  ∀ ?2 3.3.5Generalisation.The final generalisation phase takes all the unsolved meta-variables and constraints and prepends them to the type signature.To explore why some constraints and meta-variables may be unsolved (excluding those introduced for input and output constraints in the previous step) consider the following (not very useful) definition: At the end of the constraint solving phase, the type-checker will be in the following state: g : Bool ?0 ?1 -> Bool ?0 ?2 Constraints g x = not  NegPol ?1 ?2 None of the meta-variables nor the NegPol constraint itself can be solved as the type-checker doesn't know what the input linearity and polarity annotations are.In order to obtain the most general expression, we follow the constraint propagation approach inspired by the implicit generalisation feature of Idris [7].First of all any unsolved constraints are appended to the front of the type: g : {{NegPol ?1 ?2}} -> Bool ?0 ?1 -> Bool ?0 ?2 g x = not  Next we replace each unsolved meta with a new universally quantified type-variable: to result in the final function.It should be no surprise that in this case we have re-obtained the original type of not from Section 3.3.2.
Of course, in the above example we have been ignoring the function input and output constraints added in the previous phase, so the true final type is:

Worked Example
To illustrate all five stages of type-checking, we take the equalExceptAt declaration from Example 2.1: Note that for clarity, we use the high-level syntax rather than the elaborated intermediate representation.Under the assumption that the type-synonym Input has already been generalised as described in Section 3.3.1,then the first step is to insert meta-variables for the missing linearity and polarity annotations in the user's types as follows: equalExceptAt : Index 5 -> Input ?0 -> Input ?1 -> Bool ?2 ?3 equalExceptAt i x y = forall . !=  =>  ! ==  ! In this case, the function body is not modified at this stage, but if it had contained user supplied type annotations, e.g., in binders, then these would also have had fresh meta-variables inserted.
The second step is the bidirectional type-checking pass.The types that are assigned to the subterms of interest during this phase are as follows: At the end of this stage, the following meta-variables and constraints remain unsolved: : {?0, ?1, ?4,?7}Constraints : {MaxLin ?0 ?1 ?7,MaxLin  ?7 ?2}Type : Index 5 -> Input ?0 -> Input ?1 -> Bool ?2 The polarity of the output can be resolved to  , as there are no quantifiers with infinite domains.However, the linearity of the output, ?2, can't be directly solved for because the linearity of the inputs ?0 and ?1 are unknown.The unsolved constraints encode how these three unknowns are related.Note that if we added an extra rule that encodes the fact that  is the bottom element of the total order, we could also get MaxLin ?7 ?2 to reduce to the constraint ?7 ∼ ?2.However, this optimisation is unnecessary for the correctness of the algorithm.

Using the Analysis
Once the entire program has been type-checked then the types can be used to decide whether or not a specification is compilable to QF_LRA.In particular, any property with the type Bool   where  ≠  and  ≠  may be compiled.In Section 4 we will show that this is sound.On the other-hand if  =  or  =  then the constructed provenance attached to the  or  types, as described in Section 3.3.4,may be used to construct a suitable error message for the user.
However, as with any type-system that seeks to analyse semantic properties of a program, the analysis is inherently incomplete.For example, the following will be labelled as non-linear even though the actual result is constant: Appendix C contains a more complete list of the sources of incompleteness.
One consequence of the incompleteness is that if the analysis is used before normalisation then there exist specifications that belong to QF_LRA that our analysis will incorrectly identify as not belonging to it.There are two different philosophical approaches to this.The first, hard-line, approach is that users should not be writing such specifications.They are inherently fragile, and as our language is strictly more expressive than the language of the underlying solver, such a specification can always be rewritten to a form such that the analysis succeeds.
The second, more forgiving, approach is to use the analysis as a diagnostic procedure and only perform it on the original specification after normalisation has encountered a problem.This means that no valid specifications will be rejected, but at the cost that the error messages will sometimes be inaccurate.

A Possible Alternative Approach
In Section 1.1, we mentioned how Z3 and other tools detect such linearity and polarity problems by first normalising the specification.The main advantage of the normalisation approach is that, unlike ours, it is complete.However, as discussed in Section 1.1, it discards much of the structure present in the user specification, without which it is impossible to recreate useful error messages.
One obvious modification that would allow it to generate useful error messages, while retaining completeness, would be to augment each node in the normalised AST with the history of how the node was generated.This is roughly equivalent to dynamically computing the proof term in Figure 7 that our approach computes statically.When a problem was encountered, this data could then be used to reconstruct a useful error message.
However, there we argue there are still two main drawbacks to this approach.Firstly, storing the computation history for each node would have a memory requirement that would be at least linear in the length of the execution path.We hypothesise that this would be prohibitive for very large specifications with significant amounts of intermediate computation.In contrast, our type-based approaches' memory requirements are proportional to the size of the syntax of the user's program rather than the length of the execution paths.
Secondly, the normalisation approach is not compositional, i.e. it requires the whole program to be available to be run and normalised.Therefore, it cannot be used to identify linearity and polarity problems in incomplete programs or external libraries.In contrast, type-based algorithms are inherently compositional.Therefore, our analysis can be run on incomplete programs and external libraries can be checked for problems independently of the user's own code.

SMT Translation by Evaluation
We have described how to use types to identify specifications that can be translated to SMT solvers that handle linear constraints and uninterpreted functions.We complete the journey by using the inferred typing information as input for a Normalisation by Evaluation (NbE) procedure that turns a higher-order specification from Section 3 into input suitable for an SMT solver.The results of this section have been formalised in Agda2 .To simplify our development, we assume that all higher kinded types have been fully applied (this is guaranteed by the elaboration process) and that we only deal with queries in existential form (universal and parallel queries would be minor extensions).
Due to the relatively high complexity of the IR -indexed types, polymorphism, arrays, and higher-order functionswe use Categorical Logic techniques to build three models that constitute our normalisation procedure and correctness proof.The first model in Section 4.2 defines the "standard semantics" of IR, i.e., where the Real type is interpreted as rational numbers, unquantified Booleans are interpreted as Booleans, and quantified Booleans are interpreted as Agda Sets.The second model in Section 4.3 defines a semi-syntactic "normalising" model, where linear Reals are interpreted as linear expressions, unquantified Booleans as constraint expressions, and quantified Booleans as logical formulas with quantifiers.Interpreting a specification in the normalising model, and evaluating it inside Agda, yields a logical formula suitable for an SMT solver, exploiting typing information to ensure that normalisation is always possible.
The key challenge in constructing the normalising model is to handle the two tricky features of the language that make normalisation non local: the polymorphic if-then-else, and the separation of those constraints involving function terms and those containing linear arithmetic.Translation of these features requires them to be "lifted" out of their context and translated into additional constraints in the final query.We accomplish this via a specially designed monad.
The correctness criterion for our normalisation procedure is that the standard semantics of a specification  agrees with the semantics of the normalised form of , in the sense that they are equi-inhabited Agda Sets.This means that if the SMT solver a model for the existential quantifiers in a query, then it is possible to translate these back to the quantifiers in the original specification.
To prove correctness, we define a third model of the language that relates the standard and normalising semantics.This is effectively a Kripke logical relation between the two models, defined as a model itself.Our use of categorical logic to construct this model means that most of the effort is concentrated on the specifics of dealing with linear arithmetic and logical constraints with quantification.

Representing the Syntax
We represent the syntax of the IR in Agda using an intrinsically typed representation [1,4,6].This means that we define a datatype Δ | Γ ⊢  of well-typed terms, where Δ is a kinding context assigning kinds to type level variables, Γ is a typing context assigning types to term level variables, and  is the result type.The constructors of this type include the standard typed -calculus rules for type and term level abstraction and incorporate the typing rules given in Figures 3 and 4. Variables, at both the type and term level are represented using de Bruijn indices.We not require renaming and substitution at the term level, but we do at the type level in order to interpret universal quantification.

Standard Interpretation
The "standard interpretation" of the IR interprets the calculus as if it were making statements directly in mathematics.We interpret types as sets and terms as functions from the interpretation of contexts to the interpretation of the result type.We interpret Real  for all  as the set of rational numbers and the operations of addition and multiplication as the actual operations.Due to our use of Agda as our metatheory, we are led to be a little more refined in our interpretation of Bool.For Bool  , we use the set of Boolean values.Using Booleans allows us to interpret the if then else construct as an actual if-then-else.For all other polarities , Bool  , the natural interpretation would be to interpret the type as an Agda Set and interpret universal and existential quantification using Agda's Πand Σ-types.However, this leads to a universe level mismatch with the interpretation of numbers and unquantified Booleans (rationals and Booleans are at level 0, while Set is at level 1) which requires noisy explicit lifting the interpretation.Therefore, we interpret Booleans with quantifier polarity using codes for the quantifiers and translate into Set for complete programs.
The standard semantics defines a function from closed terms of Boolean type to sets, parameterised by the interpretation of any uninterpreted functions in the term.The informational content of these sets is the values in the quantifiers used in the specification term.

Normalising Semantics
The key idea behind NbE, in general, is to provide a semantics in the syntax of normal forms, rather the "intended" meaning of the calculus.In this case, we interpret the Real type as linear expressions and the Bool type as constraint expressions, then, and interpret all the operations of the calculus as ones that manipulate expressions in normal form simulating the intended meaning.Consequently, evaluating a closed term ⊢  : Bool  ∃ will yield a normal form constraint expression suitable for a solver backend which is guaranteed to have the same meaning as .This technique as similar to the use of "smart constructors" for maintaining normal forms in typed functional programming.

Context Indexed Sets.
Normal form expressions' free variables may not be the same as those of source language expressions.For example, a source language variable of type Real  will be a rational value, not a variable, and normal form expressions will only ever have rational-valued variables.To track free variables in normalised terms, we use linear variable contexts defined by the following grammar: References to variables in a linear variable context are represented as de Bruijn indices counting into the context: data Var : LinVarCtxt → Set where zero : Var (Δ, •) succ : Var Δ → Var (Δ, •) Normalised syntax will seldom be used in the context where it is created.To move syntax between contexts, we rename the variables in them.A renaming from Δ 1 to Δ 2 is a mapping of variables in Δ 2 to variables in . Each of the different kinds of syntax used in the normal forms must be renamable, meaning that if  : LinVarCtxt → Set is a set indexed by linear variable contexts, and Δ 1 ⇒  Δ 2 is a renaming, then there is a function from  Δ 2 to  Δ 1 (note that renaming is contravariant, because it acts on contexts).

Normal
Forms.We use the following Agda data type to represent linear arithmetic expressions in normal form.This type is indexed by a linear variable context Δ that controls what variables may appear.The form of the data type ensures that all LinExp Δ terms consist of trees of terms added together, with constants or variables multiplied by constants at the leaves.
data LinExp (Δ : LinVarCtxt) : Set where We adopt a convention of using backticks ' to mark constructors used for normal forms.Constraint (unquantified Boolean) expressions are represented by another data type: The first four constructors represent equality, disequality, and inequality constraints between linear expressions.The fifth and sixth constructors represent equality and inequality constraints between variables and the uninterpreted function applied to variables.We separate these two kinds of basic constraint for two reasons.In general SMT solving, the underlying DPLL(T) procedure only deals with individual constraints from pure theories, with communication between them handled by a Nelson-Oppen theory combination.So at the lowest level, the two different kinds of constraint must be separated.The second reason is that in the Marabou solver for neural networks [14], the uninterpreted function is instantiated with an actual neural network at solving time, and Marabou's input format requires references to the input and output variables of the network to be separate from the actual application of the network, which is left implicit.The final two constructors allow combination of Constraints via conjunction and disjunction.
Both the LinExp and Constraint data types enforce a normal form.LinExps push all multiplication by constants to variables, and Constraints represent constraints in negation normal form.Counterparts to the operations of multiplication and negation directly manipulate the syntax trees to maintain these normal forms: We construct normalised formulas with existentials in them in two stages.First we generate "tree" formulas with constraints at the leaves and quantifiers anywhere, as described by the following data type: We flatten the ExFormula representation to a formula in prenex form, where all existential quantifiers are at the top level.This representation can be submitted to an SMT solver.

Lifting Monad.
With these representations of normal forms, we can nearly define an NbE procedure for our language.However, there are two features of our IR that disrupt a straightforward NbE algorithm.
1.When normalising if  then  else  conditional terms we will no longer get a Boolean result from evaluating the condition , instead getting a Constraint.If  and  are themselves Constraints, then we can apply the meaning-preserving transformation if  then  else  (( ∧ ) ∨ (¬ ∧ ) (using the constructors and function defined in the previous section).However, if  and  are both linear expressions, or both of function type, then it is not clear how to proceed.Intuitively, we will always be able to translate away conditionals because a complete property is of type Bool  , but this relies on knowledge of the context in which linear expressions and functions are interpreted.2. A similar problem occurs when attempting to interpret applications of uninterpreted functions into the linear expression syntax, which does not contain function applications.An uninterpreted function application needs to be interpreted in the wider context of a constraint expression in terms of existentials and constraints.For example, a use of   in a constraint will be translated to ∃ .∃. =  ∧  =   ∧ , where  is used as the replacement for   in the translation of the context into .So we effectively need the ability to make new definitions for variables as we interpret.To address these features, we enrich our interpretation to pretend that we do have them by treating them as computational effects that can be handled in a context where they can be interpreted.We define a monad Lift on the category of LinVarCtxt indexed sets that supports operations for interpreting conditionals and definitions.If we define the interpretation of Real as not just LinExp but Lift LinExp, then when interpreting terms of type Real we are able to pretend we have conditionals and let expressions.By applying Moggi's Call-by-Value [19] translation to systematically extend our interpretation to use Lift everywhere, we will be able to interpret conditionals and definitions at any type.The normalising interpretation of a closed term of existential Boolean type will live in Lift ExFormula, whereupon we will be able to translate conditions and let expressions into Boolean combinations and existentials, respectively.
The carrier of the Lift monad is defined as an indexed inductive type.Each constructor implicitly quantifies over all linear variable contexts Δ. data Lift ( : Set  ) : Set  where return The letFunexp and letLinexp constructors extend the context with the result of a linear expression or function application respectively.They can be thought of as the moral equivalents of let  =  in  ′ .The if constructor represents a conditional predicated on a Constraint, with true and false branches as children.The return constructor terminates a piece of virtual syntax with some linear variable context indexed value.

4.3.4
The Normalising Interpretation.Equipped with the normal form types LinExp and ConstraintExp and the lifting monad Lift, we can now define the normalising interpretation of the language.Here we use the linearity and constraint information of well typed specifications to guarantee that an expression of type Bool  ∃ always yields a formula containing only existentials over linear constraints and functional (dis)equalities.
All types are interpreted as renamable sets as defined in Section 4.3.1.The interpretation of the -calculus part of the system is interpreted as a standard Kripke ("possible worlds") semantics [18], albeit with the Lift monad inserted in a CBV style.The base types are interpreted like so, using the linearity and polarity information: Note that non-linear numbers are represented by the unit type ⊤.Any attempt to multiply linear expressions will result in the unique value of this type.Since, by the typing relation, it is not possible for non-linear terms to appear in constraints, this discarding of information has no consequence.The interpretation of a whole program in the normalising semantics, after expansion of Lift ExFormula into an ExFormula and flattening to prenex form, is a function: Thus, just by our intrinsic typing of source and target programs, we know that we have a total function for normalising programs that is always type correct.

Correctness
We are guaranteed well-formedness of normal forms by Agda's type checking.To prove full correctness, our desired result is that for closed programs  : ⊢ Bool  ∃, the standard semantics, for all function interpretations  , S   (a set) and the normalising semantics N  (a formula) agree, in the sense that the set and the semantics of the normal form are equi-inhabited.This specification only covers the Bool type, however, so we define a logical relation to lift this property to all the types in the language.This logical relation relates the standard semantics Section 4.2 and the normalisation semantics Section 4.3 at all types.The structure of the logical relations follows the Kripke semantics of the normalising semantics closely.To show that evaluating the syntactic formula from the normalisation yields the same result as the standard semantics, we move from sets indexed with linear variable contexts to sets indexed by a pair of a linear variable context and an environment mapping variables to concrete rational values.Once this setup is established, the construction of the logical relations proof is structured as an alternative interpretation of the syntax, reusing the general notions from Kripke semantics.
Theorem 4.1.For closed terms  : ⊢ Bool  ∃, the standard semantics and the interpretation of the normalising semantics are equi-satisfiable, for all concrete interpretations of the (syntactically) uninterpreted function  : S   ⇔ N

Conclusions
We have presented a type-system for higher-order specification languages that tracks whether a specification can be compiled to a QF_LRA solver.A key feature of this type system is that the user can program as if it doesn't exist; they are not required to think in terms of the constraints of the underlying solver when writing their specification.Our system is constructive in two senses: if it detects that the specification may not be compilable, it constructs a detailed error message to guide the user to the source of the problem; if the system can prove that a specification is compilable, the type information generated is used directly in a normalisation procedure to generate formulas in a form suitable for a QF_LRA solver.Moreover, we have proved that the normalisation procedure is total, type preserving, and semantics preserving in Agda.
Our system is necessarily incomplete in terms of which specifications can be compiled, as we discussed in Section 3.5.To remove all sources of incompleteness in general appears to require some form of dependent types in order to track the semantic meaning of terms.The alternative, discussed in Section 3.5, is to delay checking until after an error has occurred in normalisation.The result of this would be that all valid specs would compile, at the cost of the error messages sometimes being inaccurate.
Our normalisation procedure is an example of logical encoding of a high level language into SMT form.Solvers such as Z3 [11] and CVC5 [3] perform some encoding internally, e.g., to separate constraints between theories as we did for uninterpreted functions and linear constraints, but make no formal claims of correctness for these.Tools such as SMTCoq [12] embed solving into a higher order language, but assume that the problem is in the right form for a solver before it is processed.Our procedure is remarkably generic in that most of the technical development is concerned with general concepts common to all Kripke semantics, and we expect that it will be applicable to backends beyond SMT solvers.We intend to parametrise over the choices of base kinds and types to develop a general framework for normalising DSLs.An interesting DSL is the Nested Relational Calculus (NRC) which, like our language, is a higher order language that normalises to flat SQL queries.Normalisation proofs for NRC in the past have been significantly complicated by the presence of if-then-else (e.g., Cooper [8], which had a bug fixed by Ricciotti and Cheney [23]).Our Lift monad (Section 4.3.3),provides a generic method.
The system we have presented is currently in use in the Vehicle tool for neural network verification [10].As the use of DSLs backed by automatic theorem provers is only going to increase, we hope that this work may be of use to others in improving the user experience for the next generation of solvers.

A Error Messages
A.1 Other Tools Z3 SMTLib: The following non-linear SMTLib program: (set-logic QF_LRA) (declare-const a Real) (define-fun f ((x Real) (y Real)) Real (* x y)) (assert (>= (f a a) 0) (check-sat) when run with Z3 version 4.8.12 outputs the error: (error "line 5 column 0: logic does not support nonlinear arithmetic") In this case Z3 correctly identifies that the problem is the non-linearity, but doesn't provide any information about where the non-linearity is.In big specifications with many hundreds of lines, the non-linearity can be very hard to find.Note: 'canceled' or 'resource limits reached' means the SMT query timed out, so you might want to increase the rlimit; 'incomplete quantifiers' means Z3 could not prove the query, so try to spell your proof out in greater detail, increase fuel or ifuel 'unknown' means Z3 provided no further reason for the proof failing (see also Test.fst (5,46)) This error message is problematic for two reasons.Firstly, the suggested fixes of increasing the resource limit or the fuel wouldn't work.Secondly, while it does mention the general problem 'incomplete quantifiers', this phrase may not be meaningful to a non-expert user who does not understand the Z3 internals.Furthermore, it doesn't identify the alternating quantifiers as the source of the incompleteness.
We note that in general F* does support alternating quantifiers, via SMT patterns.Nonetheless, in this case no such patterns exist and therefore we believe the error message should refer to the alternating quantifiers as the problem.

A.2 Our Error Messages
We now present example error messages generated by our analysis.gives the error message: Cannot verify specifications with alternating quantifiers.In particular: 1. the inner quantifier is the 'exists' located at line 4, columns 22-28 2. which is turned into a 'forall' by the 'not' at line 4, columns 18-21 3. which is turned into a 'exists' by the 'not' at line 4, columns 14-17 4. which alternates with the outer 'forall' quantifier at line 4, columns 5-11.Cannot verify specifications with non-linear constraints.
In particular the multiplication at line 4, columns 14-15 involves: 1. the quantified variable 'x' introduced at line 7, columns 5-11 2. which is used as an input to the function 'square' at line 7, columns 27-35 3. which is used on the left hand side of the multiplication and 1. the quantified variable 'x' introduced at line 7, columns 5-11 2. which is used as an input to the function 'square' at line 7, columns 27-35 3. which is used on the right hand side of the multiplication Cannot verify specifications with alternating quantifiers.
In particular: 1. the inner quantifier is the 'forall' located at Line 4, Columns 14-20 2. which is returned as an output of the function 'g' at Line 10, Columns 23-31 3. which is used as an input to the function 'notApp' at Line 10, Columns 16-33 4. which is turned into an 'exists' by the 'not' at Line 7, Columns 12-15 5. which is returned as an output of the function 'notApp' at Line 10, Columns 16-33 6. which alternates with the outer 'forall' at Line 10, Columns 5-11

B Constraint Solving
The subtyping relation (Figure 8) encodes the notion that the linearity and polarity of a type is the maximum of a set of other similar types's linearities and polarities.For example, [Real , Real ] are subtypes of Real  as their linearities are bounded above by .In contrast, [Real, Real ] are not a valid set of subtypes of Real  as  is above  in the lattice defined in Figure 6.The notation Bool _ _ ∈  indicates that at least one of the types in  is of type Bool   for some  and .Rules are tried in order.Specific subtyping rules only need to be defined for the types with polarity and linearity annotations (i.e.Real and Bool) and for the Vec which may recursively contain such types.The remaining types are related by equality in the final rule.
Why is the Subtypes relation n-ary?A binary relation would be infeasible as it would not allow for the calculation of a tight upper bound.A ternary relation is minimal, and indeed the n-ary relation generates ternary MaxLin and MaxPol constraints.The transition point from n-ary to ternary is in some sense arbitrary, but our choice to do it here greatly simplifies the type of vector literals.We have not implemented rules for function subtyping, but we hypothesise it could be done following the extensive literature in this area.
The solutions for HasExists (Figure 9) are analogous to those for HasForall presented in Figure 5 in Section 3.3.3 in the main text.
Equality (Figure 10) is overloaded to apply to Real, Index and Vecf the above.Note that extending equality to the Bool would necessitate splitting the type class HasEq into two separate classes, one for equality comparisons and one for inequality comparisons.This is because the latter implicitly involves a negation and therefore would affect polarity.
Ordering (Figure 11) is restricted to only apply to the Real and Index types.
Finally, the HasIndexLiteral constraint simply ensures that the literal value fits inside the required index type (Figure 12).

C Incompleteness
As discussed in Section 3.5, the type-system is incomplete and computes an under-approximation of whether a given specification lies in the QF_LRA fragment.Here we list of some of the sources of incompleteness, and a brief discussion of whether they can be mitigated.

Numeric cancellation.
As no normalisation is performed, then expressions that cancel will be given a conservative classification e.g.forall .x * x − x * x >= 0 will be marked as non-linear.This is not easily fixable under the current scheme.
Boolean cancellation.Similar to numeric cancellation, the following: False and exists . >= 0 will be marked as existentially quantified.This is not easily fixable under the current scheme.
Vector indexing.Thanks to the subtyping relation in Appendix B, a vector of rationals takes the maximum linearity of its elements.Therefore if x is a quantified variable then [, 0] will be typed as a vector of linear rationals.Consequently: forall .[x * x, 0] ! 1 >= 0 will be typed as a non-linear expression despite actually being constant.To solve this, the type-system could be refined to have dependently-typed indexing of heterogeneouslytyped products.

If-statements.
A similar problem occurs in the different branches of if-statements, e.g.

Figure 2 .
Figure 2. Intermediate representation for the language in Figure 1.

Figure 3 .
Figure 3. Type system for the builtin constants in the intermediate representation.

Figure 4 .
Figure 4. Type system for the literal expressions in the intermediate representation.

Figure 5 .
Figure 5. Rules for solving the HasForall type-class constraint

3. 3 . 4
Function Application Constraint Insertion.An observant reader may have noted that so far the constraints InputPol, OutputPol, InputLin, OutputLin in the grammar in Figure 2 have neither shown up in the types in Figures 3 & 4 nor in the constraint solutions in Figure