Topics 1: Mathematical knowledge representation. Coding theory

Lecture 6

Mathematical Knowledge Representation

From a cognitive point of view, one goal of CA is to represent algebraic knowledge in a form which can be processed and manipulated by computer.

One conceivable approach is considering different types ok knowledge: theoretic, tabulated, specification, algorithmic, deduced and strategic knowledge.

Theoretical and tabulated knowledge

Theoretical knowledge is often given in form of mathematical formulas and theorems. They affect the choice of suitable data structures and

algorithms in CAS, especially in terms of specifications and verification of the system. In general, this knowledge is neither a formal part of a CAS

nor is it recognizable from the outside. Exceptions are formalized mathematical propositions within

a logic calculus, or formulas stored in data bases. To date, such interconnection to data bases is realized in very

few CAS, thus tabulated knowledge was hardly used in this context so far. Examples for application of tabulated knowledge are

implementations of special functions, or orthogonal polynomials.

Specification knowledge Comprises the formal description of abstract data types used in the system. It contains the definition of signatures for all operators and relations of the data

structures under consideration. Furthermore, the specification of corresponding concrete computational structures

has to be added. It includes declarations for certain mathematical properties of the structures;

properties which are also used for automatic verification by type checking. The treatment of algebraic data types in CAS raises the following question:

for correctly programming mathematical procedures, implementation of new data types is often unavoidable, because algebraic algorithms in generic form are applicable only to certain data types.

Unfortunately, most current CAS offer only predefined data types. Specification knowledge can be realized in the framework of algebraic

specification. One such attempt can be found in Axiom,

where so-called categories represent abstract, and domains represent concrete data types respectively.

Properties of operators, which should specify the semantics of the operators, can be defined, however they are not checked by the system later on.

A conceivable approach for handling properties of operators is the application ok knowledge representation formalism whose semantics is clearly defined.

Algorithmic and deduced knowledge

Algorithmic knowledge is at very core of CAS. An algorithm is realized by procedures and programs. It means the processing of predominantly theoretic knowledge, but also

specification knowledge. The set of programs finally contains the algebraic knowledge for

solving mathematical problems. Deduced knowledge is used to obtain new propositions from a set of

given propositions (formalized theoretic knowledge), and general inference rules. Most of the time it is confined form of knowledge representation (e.g.

logic calculus). One could argue that deduced knowledge combined with (formalized)

theoretic and specification knowledge already constitutes an expert system.

In such systems, the search for solutions can be represented as search in a decision tree.

So far, this kind of strategy does not seem to be very promising, without the aid of additional heuristics.

Constrained Logic Programming

CLP is an approach which combines algorithmic knowledge with deduced knowledge.

In the case of CA, the process of syntactic resolution, as used, e.g. in Prolog, is amended by semantic resolution over algebraic structures, for which decision procedures or quantifier elimination methods are known.

In CLP systems, the validity of verified in a special CA part of the program, and depending on the result, the resolution process continues.

Strategic knowledge The heuristics, or rules of thumb, are part of strategic knowledge. That is where experience of users and experts becomes integrated into

the system. Strategic knowledge can be found in many programs; for example, a

procedure for factoring integers would perform trial divisions, before it switches to more complicated methods.

A procedure for symbolic integration tries to apply simple heuristic first, Risch’s algorithm is used as a last resort.

Additionally, tabulated knowledge in the form of integral tables come into play again.

One has to recognize that accessing such data bases is nontrivial task, which requires application of involved techniques like unification and pattern matching.

In 1967, heuristics has been already used in Soldier system; in 1990 in DELiA system.

Another example for appropriate exploitation of heuristics is solving differential equations exactly, again because pure algorithmic methods are in general very time consuming

CAS and expert systems The basic architecture of current expert systems can be sketched as follows:

the two major parts which make up the system are on the one hand, a knowledge base, containing the knowledge about a subject, and on the other hand, an inference engine, containing deduced knowledge as

well as strategic knowledge via application of heuristics, for solving problems. Part two in particular, has only rudimentary implementation in CAS. Macsyma was often cited as an expert system from within the artificial

intelligence community. A reason for acquiring this reputation might be that Macsyma used to be the

most extensive CAS at the time of its creation. In addition, the original draft of the system was motivated by the results of

artificial intelligence at that time. However, its structure differs, with regard to its architecture, from what we would

call an expert system nowadays. If we use the characterization in the strict sense, then none of the current CA

systems would qualify. There are also projects to use CAS, and symbolic computing in general, as test

platform for artificial intelligence methods. Examples to that end are the LEX and Meta-LEX, and some experiments suing

the integration package of Reduce

Abstract Data Types The concept of data types arose more than 25 years ago, in response to

a crisis in software development. It was intended to contribute mathematical methods to efforts targeted

towards producing reliable (and safe) software both faster, and more cost-efficiently.

To that end, the mathematical semantics of programming languages were introduced, as well as the consequent application of abstract data types during specification and implementation of programs.

An abstract data type consists of specification of its operators, and a set of axioms (i.e. equations) defining the properties of those operators.

The theory of algebraic specification, which had its origin in universal algebra, defines an abstract data type to be the initial model within the equational theory given by axioms; more succinctly, it is the isomorphism class of all initial algebras of that

equational theory. Using algebraic specifications techniques, problems such as specification,

correctness, or error handling can be treated exactly. Algebraic specification could make reusing software easier, a property

which is of prime importance, not only for CA.

Abstract Data Types Abstract data types are a powerful tool for developing algorithms;

They realize the major concepts in program development, namely separation and abstraction, in an ideal way.

Algebraic structures, such as rings, are naturally suited for representation by abstract data types.

In practical applications however, data types which are typical for computer algebra do often not allow for efficient computing.

In order to still be able to design efficient CAS based on abstract data types, one must weaken the rigorous definition (specification of operators plus equations) somewhat: For example, in Axiom, properties such as associativity of functions are

described by attributes instead equations. These declarations have to be heeded by programmers, although they can

neither be enforced nor verified by the system. Via abstract data types, a type system including parameterized types, and

types of higher order (types of types) cab be realized in a natural manner. In Axiom this concept of types is able to express algebraic structures

directly: the type Polynomial is of type Ring, and has one parameter, namely the type of its coefficients (also of type Ring).

Abstract Data Types Parameterized types enable one to write generic functions.

For example a function which operates on objects of type Polynomial has to be written only once, using the arithmetic operations of the underlying coefficient domain.

While strong typing is a considerable advantage (modularity, error avoidance, and easier debugging) during algorithm design, the situation for interactive user interfaces is totally different: users should not, as far as possible, have to deal with complicated type information. They should be able to enter formulas the way they are used to.

The CAS then has to employ some mechanism for type inference in order to invoke a suitable and mathematically correct function. The process of inference is based on coercion.

To illustrate the required steps, let consider the basic expression 3/7+x a system with type inference is able to determine the type of 3/7 as

rational number, using a bottom-up process. Next, it can not just add the symbol x to that number. The system will rather infer that both x and 3/7 can be converted

(coerced) to type Polynomial(Fraction(Integer)). Finally, it will add the two polynomials.

Coding Theory Coding theory and cryptography are branches of communication

technology parts of which are close to algebraic theories. As a result, CA tools and methods can be used for supporting

testing conjectures and deriving efficient algebraic algorithms. Coding theory deals with the problem of reliable transmission of

information through unreliable channels. The theory of error-correcting codes has been established itself

as an independent mathematical discipline with various connections to other areas such as combinatorics, lattice theory, cryptography, algebraic geometry, computational complexity, graph theory, or computational algebraic number theory.

Coding Theory Applications of CA to coding theory can be divided into two categories:

applications at an experimental level, and applications at a conceptual level.

In the first category: CAS are used to test conjectures by designing examples that are larger than those

that can be constructed by hand. The data to gathered may form the basis for the proof of a general result, or may

lead to counterexamples. For example, the problem of distribution of Hamming distances of a linear code can

be attacked by CA methods which exploit symmetries inherent to the problem. At a conceptual level,

CA is used for solving problems from computational coding theory. As example one can consider the applications to the theory of algebraic geometric

codes which are constructed by evaluating certain functions on an irreducible algebraic curve at the rational points of the curve.

When the underlying curve is the projective line, one obtains the class of Reed- Solomon codes used in many communication situations like satellite and deep-space communication or CD-players.

The construction of algebraic-geometric codes calls for methods from CA, as one has to construct bases for linear divisors on the underlying curve.

On the decoding end, one can use methods from linear algebra or Groebner basis algorithms.

Factorization of multivariate polynomials over finite fields is an essential tool for efficient decoding of algebraic-geometric codes.

Quantum Coding Theory

The prospect of quantum algorithms outperforming classical procedures gave rise to the question of realizability of quantum computers.

Most interesting from the computer algebra point of view is the ongoing classification of quantum codes via polynomial invariants of the so-called group of local unitary transforms.

Invariants of this group are constructed via matrices which are invariant under conjugation with the operation induced on the monomials.

Cryptography Modern cryptography employs a lot of techniques from computational

mathematics, most prominently ones from algebra and number theory. Apart from the well-known public key schemes which are based on the

difficulty of factoring (RSA), calculating the discrete logarithm, and decoding a linear code, there are schemes relying on computational problems like finding a closest vector in an integer lattice or solving algebraic equations.

Some of the proposed algebraic schemes for public key cryptography can be broken using algebraic techniques.

Breaking ciphers, i.e. cryptoanalysis, is another area where algebraic techniques can be applied successfully.

Methods of digital signal processing, correlation analysis, and linear algebra with added stochastic components provide the basics for investigating the asymptotic behavior in large scale experiments.

The algorithm for determining the linear complexity of a shift register sequence is an example of a typical computer algebra application.

A stream cipher uses a cascade of linear feedback shift registers each one stepping up the next. This stop and go generator was analyzed using eigenvalues over the

rationals computed by Axiom

Cryptography Data structures of CA are also significant in the area of system security,

especially in authentication, protocol verification, and in shared control schemes

The major implementations of shared control schemes are based on finite geometry, where the intersection point of appropriately chosen hypersurfaces over finite fields is the shared secret.

In multiparty protocols, like electronic voting, one calculates with points on curves which are described by polynomials.

After the computation a public interpolation method reveals the result but not the data which was private to the participants.

Another cryptographic primitive becoming more important with Internet technology is secure outsourcing.

There one wants to use an untrusted resource for linear algebra computations or for computations necessary for RSA based protocols.

CA can also help to derive algorithms and chip architectures for cryptographic purposes.

Term Rewriting Systems Term rewriting is a discipline of replacing first order terms according to

given set of rule. For example, a rule +(x, x) -> twice(x) where x is a variable, can be

used to rewrite the term 8(+(−5),−(5)), +(6, z)) to (twice(−(5)), +(6, z)). A set of such rewrite rules is called a term rewriting system of TRS for

short. Term rewriting systems are onmipresent in CS.

They can be thought of as a generalization of grammars that allows to parameterize productions.

Under certain restrictions term rewriting systems qualify as a simple functional programming language.

The most popular term rewriting mechanism is probably the macro processing known from programming languages like C and C++ (#define directives).

Some computer algebra systems like Reduce or Mathematica use term rewriting as a major vehicle for computation.

Term Rewriting Systems From a more theoretical point of view the importance of term rewriting systems for the

area of CA lies in their potential to describe a canonical simplifier for many congruence relations.

That is, some term rewriting systems can be used to compute canonical (i.e. unique) normal forms

for all congruence classes of certain algebraic structures. This is what makes term rewriting systems particularly interesting in universal algebra

and logic. Computing with term rewriting systems means to rewrite an input term repeatedly

until no more rewrites are possible. The resulting irreducible term is called a normal form.

It can be shown that this computation mechanism is Turing complete. Therefore showing the termination of all possible rewrite chains for a rewriting system

does not allow for infinite rewrite chains it is called terminating. A further issue is that in general computation with term rewriting systems is

indeterministic. Confluence is an important property of term rewriting systems that implies

deterministic results, if any, through indeterministic rewriting. A term rewriting system that is both terminating and confluent always computes

canonical forms for its input terms. Such term rewriting systems are often called complete or canonical.

Term Rewriting Systems The symmetric, reflexive and transitive closure of a term rewriting relation describes a

congruence relation: the congruence relation associated with the term rewriting system.

Hence a complete term rewriting system computes canonical forms for its associated congruence relation.

A pair of congruent terms is called an equation. An important challenge is to find a complete term rewriting system for a given

congruence relation. One solution is the Knuth-Bendix completion procedure which consists of two key

steps:1. deducing a new equation (called critical pair) from two rules and2. transforming a critical pair into a rule such that the term rewriting system computed so

far augmented by the new rule is terminating. The second step mail fail causing the whole procedure to fail. Another difficulty is that the process of producing and consuming critical pairs may

run for ever. Only is this process terminates a complete term rewriting system results. Otherwise the process may be employed as an congruence test for pairs or terms:

two terms are congruent w.r.t. input term rewriting system if they have a common normal form w.r.t. a term rewriting system computed as an intermediate result of the completion procedure.

Term Rewriting Systems Rules describing the commutative law (C) or both associativity and

commutativity (AC) of certain operators always conflict with the termination requirement and therefore Knuth-Bendix completion always fails if such properties are present.

To overcome this difficulty all basic operations of term rewriting were extended to deal with finite congruence classes of terms.

This is called term rewriting modulo a finite congruence class (e.g. modulo C or modulo AC).

An equational specification consists of a set of operators (the signature) and a set of equations describing congruences over terms constructed from the operators of the signature and a countable set of variables.

From a logical point of view a rewrite rule l -> r describes a universally quantified equation for all X : l = r where X is the list of variables occurring in l and r.

Each equational specification specifies a class of algebras which are called the models of the specification.

This class of all models of an equational specification is also called the variety of the specification.

A theorem by Birkhoff states that two terms a and b are equal in all models of a set of equations e iff b can be derived from a by repeatedly applying equations from e (back and forth).

Term Rewriting Systems There are many classes of algebras that can be characterized by equational

specifications and for which canonical term rewriting systems are known: free semi-groups, free monoids, free Abelian groups, free commutative rings, Boolean rings, distributive lattices, modules, algebras and multivariate polynomials over commutative ring structures.

For example, a free group can be specified by the following implicitly universally quantified equations: 1 · x = x, x · (x)^(−1) = 1, x · (y · z) = (x · y) · z where the signature consists of the binary infix operation ·, a unary postfix operation ()^(−1) and a constant 1; x, y, z are first order variables.

The complete term rewriting system associated with this free group is:

Term Rewriting Systems If an equational specification for one of the base structures mentioned above is

augmented by a finite set of constants (0-ary operators, generators) and a finite set of equations that do not contain variables than this specification describes a finitely presented structure.

Providing the validity of an arbitrary (variable free) equation in a finitely presented structure is called the word problem.

For many such finitely presented structures term rewriting and completion can be employed to decide the word problem.

The investigation of the completion of finitely presented structures has lead to the detection of more powerful deduction rules in the completion process like symmetrization and normalized term rewriting.

For structures like semi-groups, monoids and groups the idea of term rewriting and completion can be projected to a more efficient data structures namely to strings.

This leads to string (or word) rewriting systems and completion procedures for string rewriting systems.

For example, a complete string rewriting system for the finitely presented group S =< r, t; rr; ttt, rtt > is

tT -> e Tt -> e R -> r rr ->e tt -> t rt -> Tr rT -> tr TT -> r

where R and T are inverted generators and e represents the empty string.

Term Rewriting Systems Term rewriting has become a very important technique for equational

theorem proving in automated theorem proving in first order logic. It is used intensively in theorem provers like Otter. Similar to the resolution inference the central operation of the critical

deduction is the unification algorithm. The task of a unification is to check whether an equation has a solution

and in the positive case to find a solution. Standard unification algorithms decide this problem for free term

algebras and there is at most one most general solution for each equation.

Other issues in term rewriting include: finding criteria to prove the termination of all rewrite chains w.r.t. a term

rewriting system; term rewriting systems with additional information like sorts,

conditionals, or constraints; efficient implementation of rewriting and completion; term rewriting for higher order terms

Decision Procedures for Algebraic Theories

Let K be an elementary class of algebraic structures defined by its constants, operations, and relations (i.e. by a formal language L), as well as by elementary axioms on the given objects.

Axioms are syntactically composed of formulas consisting of equations and relations of terms, the operators and, or, not of propositional logic, and the quantifiers exists and forall.

Semantically, the quantifier range over the given structures. Variables in the range of a quantifier are called bound variables, if they occur

outside they are called free variables. Formulas which contain no free variables are called sentences. Sentences correspond to true or false statements on structures in the given

language L. A decision procedure for an elementary class K of algebraic L structures is an

algorithm which takes a sentence p of the given language as input, and returns the decision whether p is true in all structures from K or not.

The decision problem for K is the question whether a decision procedure exists for K.

The decision problem is unsolvable for many classes of algebraic structures, like Z and Q, but the ordered field of real numbers, the field of complex numbers, as well as p-adic number fields, Abelian groups, boolean algebras allow for decision procedures.

Quantifier Elimination Method for Algebraic Theories A quantifier elimination method assigns to an arbitrary formula p of the given language L a

quantifier-free formula p’ having the same free variables, such that p and p’ are equivalent in all structures of the given class K, and at all evaluation points.

Because the validity of quantifier-free formula at a given point in a structure can usually be decided with ease, one obtains from a quantifier elimination method, in general, a decision procedure for the corresponding class.

This is achieved by applying the quantifier elimination method to a given sentence p first, and then deciding the resulting quantifier-free sentence p’.

In contrast to a decision procedure, which can decide sentences only, quantifier elimination methods yield a uniform decision for arbitrary formulas which depend on their parameters.

Given their generality, decision procedures and quantifier elimination methods are algorithmically very complex, as a rule at least exponentially.

Therefore, only few of these methods have practical implementations so far. Among them are decision procedures and quantifier elimination methods for reals and integers,

e.g. for: cylindrical algebraic decomposition; parameterized versions of multivariate real root computing with side conditions; automatic theorem proving and finding in real geometry; geometric reasoning about three-dimensional objects; real constraint solving in constraint logic programming; rounding and blending of solids; stability analysis for ODEs, PDEs and difference schemes; control theory; multi-objective feedback design; simulation and error diagnosis of physical networks; parametric scheduling; etc.